1. An Oral Disease Severity Score (ODSS) validated for use in oral pemphigus vulgaris
M. Ormond, H. McParland, A.N.A. Donaldson, M. Andiappan, R.J. Cook, M. Escudier, E. Hullah, J. Higham, R. McMillan, J. Taylor, P.J. Shirlaw, S.J. Challacombe, J.F. Setterfield
British Journal of Dermatology. DOI: /bjd.16265

2. Introduction What’s already known?
Cochrane review on pemphigus vulgaris (PV)
116 outcome measures, however insufficient evidence to determine which is optimal (all sites)1
Current PV disease activity indices include the recently validated2:
Pemphigus Disease Area Index (PDAI)
Autoimmune Bullous Skin Disorder Intensity Score (ABSIS)

3. Introduction What’s already known?
Oral Disease Severity Score (ODSS) developed from a multisite mucous membrane pemphigoid score3
ODSS demonstrated to be reliable in oral mucous membrane pemphigoid and lichen planus4, 5
Additionally useful in the disease monitoring in oral lichen planus6
However ODSS, PDAI and ABSIS have never been formally
validated for use in oral PV

4. Aim
The primary aim of this study was to validate the ODSS for the assessment of oral involvement in PV
The secondary aim was to compare the inter – intra- observer variability and ease of use with the PG) and the oral score in ABSIS and PDAI

5. Methods
15 patients (aged 18-80) with histologically and immunologically confirmed mucosal PV were enrolled
10 oral medicine clinicians from four UK Oral Medicine centres examined and scored patients
Each patient was scored using the:
ODSS
PDAI
ABSIS
Physician’s Global Assessment (PGA – simple score of inflammatory skin disease)
2 clinicians re-scored after a 2 hour interval

6. Methods – Oral Disease Severity Score
Site Score + Activity Score + Pain Score =
TOTAL (106)

7. Methods – Statistical Analysis (intra-class coefficient)
Inter-observer reliability
10 observers scoring all patients with each of the 4 scores
Sample size of 15 subjects required to achieve inter class coeffecient of 0.77 for inter- observer reliability
Inter class coefficient (ICC) is a measure of consistency or reliability
An ICC of 1 demonstrates perfect consistency, a score of 0.77 is considered reliable

8. Methods – Statistical Analysis (intra-class coefficient)
Intra-observer reliability
2 raters performing a replication of all scores on all 15 patients provides 80% power to detect an ICC difference of 0.5 (null value 0.2)
Fleiss and Altman’s benchmark scales of ICC used for ordinal or continuous measures
Landis-Koch’s benchmark values followed when Kappa coefficients used for categorical outcomes
These benchmarks provide a framework to rate the consistency or reliability of certain ICC values

9. Results 15 patients (f11:m4, mean age 56±14.2 years (range 23-77)
ODSS total (max 106)
Mean (SD) 22.3±12.8, range 0-68; median [IQR] 22 [ ]
Mean completion time 76 seconds (SD±37)
PDAI activity (max 90)
Mean (SD) 14.6±7.7, range 0-42; median [IQR] 14.6 [ ]
Mean completion time of 117 seconds (SD±16)
ABSIS total (max 56)
Mean (SD) 12.5±1.1, range 0-43; median 11.7 [ ]
Mean completion time 75 seconds (SD±19)
PGA (max 10)
Mean score was 6.1±0.9, (range 0-9); median 3 [1-8]

10. Results - Intra-observer reliability
Table 1a
Overall intra-observer reliability
Score
Observer 1 ICC
Observer 2 ICC
P-value
Overall*
ODSS Total
0.97
0.96
0.53
Excellent
PDAI Activity
0.99
0.82
Good / Substantial
ABSIS Total
0.86
0.45
0.46
Fair / Moderate
PGA
0.64
0.80
Table 1b
ODSS subsection inter-observer reliability
Score
Observer 1 ICC
Observer 2 ICC
P-value
Overall*
ODSS Site
0.97
0.95
0.36
Excellent
ODSS Activity
0.98
0.61
ODSS Pain
0.90
0.43
ODSS Total
0.96
0.53
*Overall ICC rating based on benchmark scales

11. Results - Inter-observer reliability
Table 2a
Overall inter-observer reliability
Score
ICC
Overall*
ODSS Total
0.83
Very good
PDAI Activity
0.79
ABSIS Total
0.71
Good / Substantial
PGA
0.70
Table 2b
ODSS subsection inter-observer reliability
Score
ICC
Overall*
ODSS Site
0.63
Good / Substantial
ODSS Activity
0.83
Very good
ODSS Pain
0.90
Excellent
ODSS Total
*Overall ICC rating based on benchmark scales

12. Discussion What does this study add?
The ODSS is shown to be a thorough, sensitive, yet quick assessment tool for oral involvement in PV
It provides a detailed assessment of 17 oral sites and excellent record of overall disease severity
Due to its superiority at assessing each oral site compared with ABSIS and PDAI, it is more sensitive to subtle changes in disease severity during sequential visits

13. Conclusions
We recommend it be considered as the standard scoring methodology for oral pemphigus vulgaris disease, to facilitate comparative outcomes and multicentre collaborative studies

14. References
Martin L, Murrell DF. Measuring the immeasurable: a systematic review of outcome measures in pemphigus. Paper Australas J Dermatol. 2006;47(Suppl1):A32–3
Rosenbach M, Murrell D, Bystryn JC et al. Reliability and Convergent Validity of Two Outcome Instruments for Pemphigus. J Invest Dermatol. 2009;129:
Setterfield J, Shirlaw PJ, Kerr-Muir M et al. Mucous membrane pemphigoid: a dual circulating antibody response with IgG and IgA signifies a more severe and persistent disease. Br J Dermatol. 1998;138:
Escudier M, Ahmed N, Shirlaw P et al. A scoring system for mucosal disease severity with special reference to oral lichen planus. Br J Dermatol. 2007;157(4):
Reeves GMB, Lloyd M, Rajlawat BP et al. Ocular and oral grading of mucous membrane pemphigoid. Graefes Arch Clin Exp Ophthalmol. 2012;250:
Wee J, Shirlaw PJ, Challacombe SJ, Setterfield JF. Efficacy of mycophenolate mofetil in severe mucocutaneous lichen planus: a retrospective review of 10 patients. Br J Dermatol. 2012;167:36-43

15. Call for correspondence
Why not join the debate on this article through our correspondence section?
Rapid responses should not exceed 350 words, four references and one figure
Further details can be found here