Modelling Cardinal Utilities from Ordinal Utility data: An exploratory analysis Peter Gilks, Chris McCabe, John Brazier, Aki Tsuchiya, Josh Solomon.

Slides:



Advertisements
Similar presentations
Elicitation methods Health care demands exceed resource supply Therefore, rationing is inevitable Many ways by which we can ration health care One is economic.
Advertisements

Lecture 11 (Chapter 9).
Welcome to Econ 420 Applied Regression Analysis
Brief introduction on Logistic Regression
Economics 20 - Prof. Anderson1 Multiple Regression Analysis y =  0 +  1 x 1 +  2 x  k x k + u 7. Specification and Data Problems.
Linear Regression t-Tests Cardiovascular fitness among skiers.
IRT Equating Kolen & Brennan, IRT If data used fit the assumptions of the IRT model and good parameter estimates are obtained, we can estimate person.
3.2 OLS Fitted Values and Residuals -after obtaining OLS estimates, we can then obtain fitted or predicted values for y: -given our actual and predicted.
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 13 Nonlinear and Multiple Regression.
McGraw-Hill Ryerson Copyright © 2011 McGraw-Hill Ryerson Limited. Adapted by Peter Au, George Brown College.
Correlation Chapter 9.
LECTURE 3 Introduction to Linear Regression and Correlation Analysis
Models with Discrete Dependent Variables
Chapter 13 Multiple Regression
Correlation and Autocorrelation
Lecture 3: Chi-Sqaure, correlation and your dissertation proposal Non-parametric data: the Chi-Square test Statistical correlation and regression: parametric.
Two ways to skin a cat: a comparison of two variants of standard gamble John Brazier and Paul Dolan Prepared for the CHEBS workshop on Elicitation, 9 October.
1Prof. Dr. Rainer Stachuletz Multiple Regression Analysis y =  0 +  1 x 1 +  2 x  k x k + u 7. Specification and Data Problems.
AN INTRODUCTION TO PORTFOLIO MANAGEMENT
Chapter Eighteen MEASURES OF ASSOCIATION
Valuing the SF-6D: a nonparametric approach using individual level preference data Part 1): The SF-6D and its valuation Samer A Kharroubi, Tony O’Hagan,
Basic Business Statistics, 11e © 2009 Prentice-Hall, Inc. Chap 14-1 Chapter 14 Introduction to Multiple Regression Basic Business Statistics 11 th Edition.
Intro to Statistics for the Behavioral Sciences PSYC 1900 Lecture 6: Correlation.
REGRESSION AND CORRELATION
Using ranking and DCE data to value health states on the QALY scale using conventional and Bayesian methods Theresa Cain.
Ch. 14: The Multiple Regression Model building
Measures of Association Deepak Khazanchi Chapter 18.
Utilising rank and DCE data to value health status on the ‘QALY’ scale using conventional and Bayesian methods John Brazier and Theresa Cain with Aki Tsuchiya.
25 Sept 07 FF8 - Discrete Choice Data Introduction Tony O’Hagan.
AN INTRODUCTION TO PORTFOLIO MANAGEMENT
ECON 6012 Cost Benefit Analysis Memorial University of Newfoundland
Nonlinear Regression Functions
Inference for regression - Simple linear regression
Simple Covariation Focus is still on ‘Understanding the Variability” With Group Difference approaches, issue has been: Can group membership (based on ‘levels.
Chapter Eight The Concept of Measurement and Attitude Scales
Why use the EQ-5D? What are the alternatives?. What are the alternatives for Direct valuation? Other VAS Time Trade-Off Standard Gamble Willingness to.
2-1 MGMG 522 : Session #2 Learning to Use Regression Analysis & The Classical Model (Ch. 3 & 4)
Health State Unable to perform some tasks at home and/or at work Able to perform all self care activities (eating, bathing, dressing) albeit with some.
Experimental Research Methods in Language Learning Chapter 11 Correlational Analysis.
Statistical analysis Prepared and gathered by Alireza Yousefy(Ph.D)
University of Warwick, Department of Sociology, 2014/15 SO 201: SSAASS (Surveys and Statistics) (Richard Lampard) Week 7 Logistic Regression I.
By: Amani Albraikan.  Pearson r  Spearman rho  Linearity  Range restrictions  Outliers  Beware of spurious correlations….take care in interpretation.
1 EQ-5D, HUI and SF-36 Of the shelf instruments…..
Lecture 8 Simple Linear Regression (cont.). Section Objectives: Statistical model for linear regression Data for simple linear regression Estimation.
Sample Size Determination in Studies Where Health State Utility Assessments Are Compared Across Groups & Time Barbara H Hanusa 1,2 Christopher R H Hanusa.
Correlation & Regression Chapter 15. Correlation It is a statistical technique that is used to measure and describe a relationship between two variables.
Discussion of time series and panel models
Lecture 14 Summary of previous Lecture Regression through the origin Scale and measurement units.
REGRESSION DIAGNOSTICS Fall 2013 Dec 12/13. WHY REGRESSION DIAGNOSTICS? The validity of a regression model is based on a set of assumptions. Violation.
Research Methodology and Methods of Social Inquiry Nov 8, 2011 Assessing Measurement Reliability & Validity.
Experimentation in Computer Science (Part 2). Experimentation in Software Engineering --- Outline  Empirical Strategies  Measurement  Experiment Process.
Basic Business Statistics, 10e © 2006 Prentice-Hall, Inc.. Chap 14-1 Chapter 14 Introduction to Multiple Regression Basic Business Statistics 10 th Edition.
Choose n out of these k objects For example:  Choose your three favorites out of these ten photographs  Of these fifty apps, which ten would you download.
McGraw-Hill/IrwinCopyright © 2009 by The McGraw-Hill Companies, Inc. All Rights Reserved. Simple Linear Regression Analysis Chapter 13.
Example x y We wish to check for a non zero correlation.
Introduction to Multiple Regression Lecture 11. The Multiple Regression Model Idea: Examine the linear relationship between 1 dependent (Y) & 2 or more.
1 HETEROSCEDASTICITY: WEIGHTED AND LOGARITHMIC REGRESSIONS This sequence presents two methods for dealing with the problem of heteroscedasticity. We will.
CORRELATION ANALYSIS.
More on regression Petter Mostad More on indicator variables If an independent variable is an indicator variable, cases where it is 1 will.
The measurement and comparison of health system responsiveness Nigel Rice, Silvana Robone, Peter C. Smith Centre for Health Economics, University of York.
Effect of framing of death on health state values obtained from DCEs Dr. Esther W. de Bekker-Grob by Jonker, de.
Inference for Regression
Lecture 7: Bivariate Statistics. 2 Properties of Standard Deviation Variance is just the square of the S.D. If a constant is added to all scores, it has.
IRT Equating Kolen & Brennan, 2004 & 2014 EPSY
McGraw-Hill/Irwin © 2003 The McGraw-Hill Companies, Inc.,All Rights Reserved. Part Four ANALYSIS AND PRESENTATION OF DATA.
M.Sc. in Economics Econometrics Module I
Chapter 12: Regression Diagnostics
POSC 202A: Lecture Lecture: Substantive Significance, Relationship between Variables 1.
Regression Forecasting and Model Building
Elicitation methods Health care demands exceed resource supply
Presentation transcript:

Modelling Cardinal Utilities from Ordinal Utility data: An exploratory analysis Peter Gilks, Chris McCabe, John Brazier, Aki Tsuchiya, Josh Solomon

Background Limitations of conventional methods of utility elicitation Early work suggesting ordinal data can predict cardinal preferences SF6D and HUI 2 surveys used ranking exercises as warm up prior to SG valuation tasks Opportunity to test and develop methods proposed by Solomon

SF-6D valuation data sets Ranked seven SF-6d states (including pits and full health) and death SG valuations of five states against full health and pits and then chained using valuation of pits against full health and death (respondents asked to confirm pits ranking against death) 611 respondents sampled from the general population 249 mean SG health states values ranging from.21 to.99; averaged 14 valuations per state

HUI2 valuation data set Ranked 9 HUI2 states (including pits and full health) and death SG valuations of 8 states against full health and death (respondents asked to confirm ranking of state against death) 198 respondents sampled from the general population 249 mean SG health states values ranging from to.77; averaged 24 valuations per state

Methods Aim: To model the predicted health state valuations using the ordinal preference data 1)Statistical model Conditional logistic regression (McFadden choice model) based on random utility theory (previous attempts used Thurstone’s Comparative Judgement Model) 2) Value function Relating the health state descriptive system to the utility value

The Statistical Model Respondent i has latent utility value for state j, U ij. Respondent will choose state j as best from a group of states k=1,…,n if U ij > U ik for all k  j. Utility function U ij = μ j + ε ij. Where μ j represents the underlying tastes of the population and ε ij represents the peculiar choice of the individual. Odds of choosing state j over state k are exp{μ j – μ k } So we want to model the dependent variable μ against the dimensions of the descriptive systems: SF6D and HUI2.

Assumption: independence of irrelevant alternatives Model is based on assumption that the ranking exercise is equivalent to the respondent making a series of individual choices from smaller and smaller groups of states. For example, to rank 10 health states; Selects first preference from all 10, rank 1 Selects best from remaining 9, rank 2 Selects best from remaining 8, rank 3 and so on…… NB. This assumes that the choice over a given pair does not depend on the other alternatives available

Value function The expected value of each unobserved utility was assumed to be a linear function of the categorical ratings on the domains of each dataset respectively. The specifications are; For HUI2: μ = β 1 S2 + β 2 S3 + β 3 S4 + β 4 M2 + β 5 M3 + β 6 M4 + β 7 M5 + β 8 E2 + β 9 E3 + β 10 E4 + β 11 E5 + β 12 C2 + β 13 C3 + β 14 C4 + β 15 SC2 + β 16 SC3 + β 17 SC4 + β 18 P2 + β 19 P3 + β 20 P4 + β 21 P5 + β d Death For SF6D: μ = β 1 PF2 + β 2 PF3 + β 3 PF4 + β 4 PF5 + β 5 PF6 + β 6 RL2 +β 7 RL3 + β 8 RL4 + β 9 SF2 + β 10 SF3 + β 11 SF4 + β 12 SF5 + β 13 P2 + β 14 P3 + β 15 P4 + β 16 P5 + β 17 P6 + β 18 MH2 + β 19 MH3 + β 20 MH4 + β 21 MH5 + β 22 V2 + β 23 V3 + β 24 V4 + β 25 V5 +β d Death Note: no constant term and a coefficient for death! This facilitates re- scaling results on to the Full-Health Death (1,0) Scale.

Rescaling The scale of the latent variable μ is arbitrarily defined by the identifying assumptions in the model. 1)Normalise to observed SG scale (originally proposed by Josh Solomon) Multiply coefficients by the ratio: β ri = β i * min. obs. SG/ Predicted PITS value 2) Normalise to death β ri = β i / |β d | This anchors death at zero and perfect health at 1 NB. states can still be valued as worse than death.

Model Assessment Methods Main aim is to compare the predictive performance of the rank model and the original standard gamble model: Check coefficients for sign and consistency. Plot predictions against observed for rank model and SG model for both datasets. Statistical tests of predictive performance. Look for systematic patterns in the errors.

HUI2

Rank Model SG Model Smooth line = mean health state values ranked by severity Top line is predictions Bottom line is error. Mean values, predicted values and error (predict - mean) for Rank model including death and SG Model (OLS) HUI2

SF6D

Rank Model SG Model Mean values, predicted values and error (predict - mean) for Rank model including death and SG Model (6) SF6D Smooth line = means Top messy line is predictions Bottom messy lines is error. Both Models: Under predict large means Over predict low means

Summary of Findings Rank models able to predict actual mean SG health states nearly as well as the SG models – associated with modest increase in in MAE Evidence that it has produced less systematic error in SF-6D data set and improvements in consistency

Issues – taking results at face value Is the ranked model good enough? Could we start using it……… Given ranking is a warm up, results could be better if more care taken over this part of the exercise Ranked methods are probably cheaper What evidence is there that ranking exercises impose a lower cognitive burden? Seems to be higher levels of completion.

Issues – harder questions Is the selection process of the ranking task assumed by the model correct? Why should the relationship between the latent utility value and SG (in this case) cardinal values be linear? –What other functional forms might theory suggest? –Is the latent utility value similar to Dyer and Sarin’s ‘value function’ or something else? Does rank data elicit preferences or simply how good or bad a health state is, and does it matter?

Issues – the death question Not a major problem here because all mean health state values above zero The MVH EQ-5D data has been analysed in a similar way by Josh Solomon, but the ranking of death was very different to the implied ranking from the TTO – only state is ranked worse than death compared to 16/43 states by TTO! Ranked model normalised to death and full health does not predict TTO values worse than death very well

Further work – more suggestions welcome See how well SG data predicts ranking at the individual level Consider interactions Model different functional relationships between latent variable and SG examine completion rates and extent to which ranking will extend the vote to more vulnerable populations