An Oral Disease Severity Score (ODSS) validated for use in oral pemphigus vulgaris M. Ormond, H. McParland, A.N.A. Donaldson, M. Andiappan, R.J. Cook, M. Escudier, E. Hullah, J. Higham, R. McMillan, J. Taylor, P.J. Shirlaw, S.J. Challacombe, J.F. Setterfield British Journal of Dermatology. DOI: 10.111/bjd.16265

Introduction What’s already known? Cochrane review on pemphigus vulgaris (PV) 116 outcome measures, however insufficient evidence to determine which is optimal (all sites)1 Current PV disease activity indices include the recently validated2: Pemphigus Disease Area Index (PDAI) Autoimmune Bullous Skin Disorder Intensity Score (ABSIS)

Introduction What’s already known? Oral Disease Severity Score (ODSS) developed from a multisite mucous membrane pemphigoid score3 ODSS demonstrated to be reliable in oral mucous membrane pemphigoid and lichen planus4, 5 Additionally useful in the disease monitoring in oral lichen planus6 However ODSS, PDAI and ABSIS have never been formally validated for use in oral PV

Aim The primary aim of this study was to validate the ODSS for the assessment of oral involvement in PV The secondary aim was to compare the inter – intra- observer variability and ease of use with the PG) and the oral score in ABSIS and PDAI

Methods 15 patients (aged 18-80) with histologically and immunologically confirmed mucosal PV were enrolled 10 oral medicine clinicians from four UK Oral Medicine centres examined and scored patients Each patient was scored using the: ODSS PDAI ABSIS Physician’s Global Assessment (PGA – simple score of inflammatory skin disease) 2 clinicians re-scored after a 2 hour interval

Methods – Oral Disease Severity Score Site Score + Activity Score + Pain Score = TOTAL (106)

Methods – Statistical Analysis (intra-class coefficient) Inter-observer reliability 10 observers scoring all patients with each of the 4 scores Sample size of 15 subjects required to achieve inter class coeffecient of 0.77 for inter- observer reliability Inter class coefficient (ICC) is a measure of consistency or reliability An ICC of 1 demonstrates perfect consistency, a score of 0.77 is considered reliable

Methods – Statistical Analysis (intra-class coefficient) Intra-observer reliability 2 raters performing a replication of all scores on all 15 patients provides 80% power to detect an ICC difference of 0.5 (null value 0.2) Fleiss and Altman’s benchmark scales of ICC used for ordinal or continuous measures Landis-Koch’s benchmark values followed when Kappa coefficients used for categorical outcomes These benchmarks provide a framework to rate the consistency or reliability of certain ICC values

Results 15 patients (f11:m4, mean age 56±14.2 years (range 23-77) ODSS total (max 106) Mean (SD) 22.3±12.8, range 0-68; median [IQR] 22 [12.75-29] Mean completion time 76 seconds (SD±37) PDAI activity (max 90) Mean (SD) 14.6±7.7, range 0-42; median [IQR] 14.6 [11.0-16.5] Mean completion time of 117 seconds (SD±16) ABSIS total (max 56) Mean (SD) 12.5±1.1, range 0-43; median 11.7 [5.4-17.5] Mean completion time 75 seconds (SD±19) PGA (max 10) Mean score was 6.1±0.9, (range 0-9); median 3 [1-8]

Results - Intra-observer reliability Table 1a Overall intra-observer reliability Score Observer 1 ICC Observer 2 ICC P-value Overall* ODSS Total 0.97 0.96 0.53 Excellent PDAI Activity 0.99 0.82 Good / Substantial ABSIS Total 0.86 0.45 0.46 Fair / Moderate PGA 0.64 0.80 Table 1b ODSS subsection inter-observer reliability Score Observer 1 ICC Observer 2 ICC P-value Overall* ODSS Site 0.97 0.95 0.36 Excellent ODSS Activity 0.98 0.61 ODSS Pain 0.90 0.43 ODSS Total 0.96 0.53 *Overall ICC rating based on benchmark scales

Results - Inter-observer reliability Table 2a Overall inter-observer reliability Score ICC Overall* ODSS Total 0.83 Very good PDAI Activity 0.79 ABSIS Total 0.71 Good / Substantial PGA 0.70 Table 2b ODSS subsection inter-observer reliability Score ICC Overall* ODSS Site 0.63 Good / Substantial ODSS Activity 0.83 Very good ODSS Pain 0.90 Excellent ODSS Total *Overall ICC rating based on benchmark scales

Discussion What does this study add? The ODSS is shown to be a thorough, sensitive, yet quick assessment tool for oral involvement in PV It provides a detailed assessment of 17 oral sites and excellent record of overall disease severity Due to its superiority at assessing each oral site compared with ABSIS and PDAI, it is more sensitive to subtle changes in disease severity during sequential visits

Conclusions We recommend it be considered as the standard scoring methodology for oral pemphigus vulgaris disease, to facilitate comparative outcomes and multicentre collaborative studies

References Martin L, Murrell DF. Measuring the immeasurable: a systematic review of outcome measures in pemphigus. Paper Australas J Dermatol. 2006;47(Suppl1):A32–3 Rosenbach M, Murrell D, Bystryn JC et al. Reliability and Convergent Validity of Two Outcome Instruments for Pemphigus. J Invest Dermatol. 2009;129:2404-2410 Setterfield J, Shirlaw PJ, Kerr-Muir M et al. Mucous membrane pemphigoid: a dual circulating antibody response with IgG and IgA signifies a more severe and persistent disease. Br J Dermatol. 1998;138:602- 610. Escudier M, Ahmed N, Shirlaw P et al. A scoring system for mucosal disease severity with special reference to oral lichen planus. Br J Dermatol. 2007;157(4):765-770 Reeves GMB, Lloyd M, Rajlawat BP et al. Ocular and oral grading of mucous membrane pemphigoid. Graefes Arch Clin Exp Ophthalmol. 2012;250:611-618. Wee J, Shirlaw PJ, Challacombe SJ, Setterfield JF. Efficacy of mycophenolate mofetil in severe mucocutaneous lichen planus: a retrospective review of 10 patients. Br J Dermatol. 2012;167:36-43

Call for correspondence Why not join the debate on this article through our correspondence section? Rapid responses should not exceed 350 words, four references and one figure Further details can be found here