1. Second Preventing Overdiagnosis Conference September 15-17, 2014 Oxford UK
 How important is Overdiagnosis to members of the public offered the chance to include it in a multi-criteria decision aid?
Mette Kjer KALTOFT* Jesper Bo NIELSEN, Glenn SALKELD, Michelle CUNICH Jack DOWIE *Research Unit for General Practice, Institute of Public Health, University of Southern Denmark, Odense 5000, Denmark

2. Objectives
To establish how often Overdiagnosis was selected from a menu of criteria by participants in an online trial of a multi-criteria decision aid for the PSA test decision for prostate cancer
To determine the relative importance assigned to Overdiagnosis by those including it
To reflect on how Overdiagnosis should be dealt with in a multi-criteria decision (MCDA) aid

3. Methods
Data are from a community trial of decision aids for having or not having PSA test
1400+ Australian males years old without diagnosed prostate cancer.
Randomly assigned to Fixed or Pick Your Own (PYO) criteria arms
Fixed arm had 5 specified criteria
PYO arm : participants selected between 1 and 10 of 10 criteria for inclusion in their MCDA-based aid
Overdiagnosis was one of the 5 additional criteria in PYO

4. Criteria (“ Avoiding… )
Labels (only Overdiagnosis has elaboration). Fixed arm criteria are in italics
LOSS OF LIFETIME
LOSS OF HEALTH
NEEDLESS BIOPSY
Avoiding OVERDIAGNOSIS (and needless treatment) as a result of a PSA test detecting a cancer that would not have affected your life or health
URINARY PROBLEMS
BOWEL PROBLEMS
SEXUAL PROBLEMS
BURDEN of TREATMENT
BURDEN to CARERS
REGRET

5. Procedures
Participants in the PYO arm selected the ones they wished to include and exclude in their aid by checking boxes in a list
Only their selected criteria appeared in their personalised decision aid screen
They then indicated their relative importance weights for their selected criteria by changing bar lengths
Longer = more important to avoid
Weights displayed not normalised to always add to 1

6. Screen with all 10 criteria (at default equal weights) Seen only by those who selected all 10 for inclusion

7. Screen with all 10 criteria and popup for Overdiagnosis criterion

8. plus his Option Scores (using normalised weights)
8% % % % % %
Option Scores
unnormalised
normalisedn
Screen for a participant who selected 6 of 10 criteria with his Weightings as entered and normalised to 100%
plus his Option Scores (using normalised weights)

9. Results 77% included Overdiagnosis in their selection
57 % of those who excluded at least one of the 10
Of those who selected 5 criteria 47% included it, as did 41% of those who selected 4 or 6 criteria
Average weights assigned to Overdiagnosis by those who included it and provided a complete set of weights increased from 9% to 19% as the selected number fell
Highest weight for Overdiagnosis was 33%

10. Conclusions
Overdiagnosis was prominent among the criteria selected from a menu by male members of the public participating in a trial of a multi-criteria decision aid
Notably, over 40% of those who excluded 4 to 6 of the 10 criteria included OD in their aid
Moderate weights were attached to OD, varying with the number of criteria selected

11. On reflection… three fundamental questions
Q1. Should OD be a criterion in an MCDA-based decision aid and if so, how should it be defined?
Q2. How should the performance of (all) relevant options on OD be performance-rated (as is necessary in an MCDA)?
Q3 Should Underdiagnosis (UD) be treated in same way as OD?

12. On reflection… response 1
Q: Should OD be a criterion in an MCDA-based aid and if so, how should it be defined?
A: Yes, but OD should be defined as a 'lifetime False Positive‘ (FP) with only the psychological and affective distress associated with being a lifetime FP included and weighted
All other consequences of over-testing and over- treatment should be included in the criteria they impact on, as with an ordinary (short-term) False Positive

13. On reflection… response 2
Q: How should the performance of (all) relevant options on OD be performance-rated (as is needed in the MCDA)?
A: In the normal way where the performance rates of the testing and treatments associated with an option are assessed by establishing the BEANs (Best Estimates Available Now) on each of relevant criteria
‘main’ effect/s
‘side’ effects, adverse events, complications…
burden/bother of test and treatment procedures per se
E.g. The performance rate of the option Surgery on the criterion Quality of Life should reflect the impact of OD on QOL in pursuing the Surgery option, not performance on the criterion OD defined as in 1

14. On reflection… response 3
Q3 Should Underdiagnosis (UD) be treated in same way as OD?
A: Yes, and/but should be defined as a 'lifetime False Negative‘ (FN) with only the psychological and affective distress associated with being a lifetime FN included and weighted
All other consequences of under-testing and under- treatment should be included in the criteria they impact on, as with an ordinary (short-term) False Negative

15. Acknowledgements Thanks for attending
Funding Sources: Mette Kjer Kaltoft’s PhD study is funded by the Region of Southern Denmark, the University of Southern Denmark and The Health Foundation (Helsefonden).The contribution of Professor Salkeld, Dr Turner and Dr Cunich was supported by the Screening and diagnostic Test Evaluation Program (STEP) funded by the National Health and Medical Research Council of Australia under program grant number
Conflicts of Interest. Jack Dowie has a financial interest in the Annalisa software but did not benefit from its use in the trial from which the data in this paper are drawn. No conflicts were reported by other authors.
Thanks for attending