Second Preventing Overdiagnosis Conference September 15-17, 2014 Oxford UK  How important is Overdiagnosis to members of the public offered the chance to include it in a multi-criteria decision aid? Mette Kjer KALTOFT* Jesper Bo NIELSEN, Glenn SALKELD, Michelle CUNICH Jack DOWIE *Research Unit for General Practice, Institute of Public Health, University of Southern Denmark, Odense 5000, Denmark E-mail: mkaltoft@health.sdu.dk

Objectives To establish how often Overdiagnosis was selected from a menu of criteria by participants in an online trial of a multi-criteria decision aid for the PSA test decision for prostate cancer To determine the relative importance assigned to Overdiagnosis by those including it To reflect on how Overdiagnosis should be dealt with in a multi-criteria decision (MCDA) aid

Methods Data are from a community trial of decision aids for having or not having PSA test 1400+ Australian males 40-69 years old without diagnosed prostate cancer. Randomly assigned to Fixed or Pick Your Own (PYO) criteria arms Fixed arm had 5 specified criteria PYO arm : participants selected between 1 and 10 of 10 criteria for inclusion in their MCDA-based aid Overdiagnosis was one of the 5 additional criteria in PYO

Criteria (“ Avoiding… ) Labels (only Overdiagnosis has elaboration). Fixed arm criteria are in italics LOSS OF LIFETIME LOSS OF HEALTH NEEDLESS BIOPSY Avoiding OVERDIAGNOSIS (and needless treatment) as a result of a PSA test detecting a cancer that would not have affected your life or health URINARY PROBLEMS BOWEL PROBLEMS SEXUAL PROBLEMS BURDEN of TREATMENT BURDEN to CARERS REGRET

Procedures Participants in the PYO arm selected the ones they wished to include and exclude in their aid by checking boxes in a list Only their selected criteria appeared in their personalised decision aid screen They then indicated their relative importance weights for their selected criteria by changing bar lengths Longer = more important to avoid Weights displayed not normalised to always add to 1

Screen with all 10 criteria (at default equal weights) Seen only by those who selected all 10 for inclusion

Screen with all 10 criteria and popup for Overdiagnosis criterion

plus his Option Scores (using normalised weights) 8% 16% 18% 19% 29% 10% Option Scores unnormalised normalisedn Screen for a participant who selected 6 of 10 criteria with his Weightings as entered and normalised to 100% plus his Option Scores (using normalised weights)

Results 77% included Overdiagnosis in their selection 57 % of those who excluded at least one of the 10 Of those who selected 5 criteria 47% included it, as did 41% of those who selected 4 or 6 criteria Average weights assigned to Overdiagnosis by those who included it and provided a complete set of weights increased from 9% to 19% as the selected number fell Highest weight for Overdiagnosis was 33%

Conclusions Overdiagnosis was prominent among the criteria selected from a menu by male members of the public participating in a trial of a multi-criteria decision aid Notably, over 40% of those who excluded 4 to 6 of the 10 criteria included OD in their aid Moderate weights were attached to OD, varying with the number of criteria selected

On reflection… three fundamental questions Q1. Should OD be a criterion in an MCDA-based decision aid and if so, how should it be defined? Q2. How should the performance of (all) relevant options on OD be performance-rated (as is necessary in an MCDA)? Q3 Should Underdiagnosis (UD) be treated in same way as OD?

On reflection… response 1 Q: Should OD be a criterion in an MCDA-based aid and if so, how should it be defined? A: Yes, but OD should be defined as a 'lifetime False Positive‘ (FP) with only the psychological and affective distress associated with being a lifetime FP included and weighted All other consequences of over-testing and over- treatment should be included in the criteria they impact on, as with an ordinary (short-term) False Positive

On reflection… response 2 Q: How should the performance of (all) relevant options on OD be performance-rated (as is needed in the MCDA)? A: In the normal way where the performance rates of the testing and treatments associated with an option are assessed by establishing the BEANs (Best Estimates Available Now) on each of relevant criteria ‘main’ effect/s ‘side’ effects, adverse events, complications… burden/bother of test and treatment procedures per se E.g. The performance rate of the option Surgery on the criterion Quality of Life should reflect the impact of OD on QOL in pursuing the Surgery option, not performance on the criterion OD defined as in 1

On reflection… response 3 Q3 Should Underdiagnosis (UD) be treated in same way as OD? A: Yes, and/but should be defined as a 'lifetime False Negative‘ (FN) with only the psychological and affective distress associated with being a lifetime FN included and weighted All other consequences of under-testing and under- treatment should be included in the criteria they impact on, as with an ordinary (short-term) False Negative

Acknowledgements Thanks for attending Funding Sources: Mette Kjer Kaltoft’s PhD study is funded by the Region of Southern Denmark, the University of Southern Denmark and The Health Foundation (Helsefonden).The contribution of Professor Salkeld, Dr Turner and Dr Cunich was supported by the Screening and diagnostic Test Evaluation Program (STEP) funded by the National Health and Medical Research Council of Australia under program grant number 633003.   Conflicts of Interest. Jack Dowie has a financial interest in the Annalisa software but did not benefit from its use in the trial from which the data in this paper are drawn. No conflicts were reported by other authors. Thanks for attending