Blackbox classifiers for preoperative discrimination between malignant and benign ovarian tumors C. Lu 1, T. Van Gestel 1, J. A. K. Suykens 1, S. Van Huffel 1, I. Vergote 2, D. Timmerman 2 1 Department of Electrical Engineering, Katholieke Universiteit Leuven, Leuven, Belgium, 2 Department of Obstetrics and Gynecology, University Hospitals Leuven, Leuven, Belgium address:

Demographic, serum marker, color Doppler imaging and morphologic variables Visualizing the correlation between the variables and the relations between the variables and clusters. Biplot of Ovarian Tumor Data 1. Introduction  Ovarian masses is a common problem in gynecology. A reliable test for preoperative discrimination between benign and malignant ovarian tumors is of considerable help for clinicians in choosing appropriate treatments for patients.  In this study, we develop and evaluate several blackbox models, particularly multi-layer perceptrons (MLP) and least squares support vector machines (LS-SVMs), both within Bayesian evidence framework, to preoperatively predict malignancy of ovarian tumors. Model performance is accessed via Receiver Operating Characteristic (ROC) curve analysis. 2. Data o: benign case x: malignant case

ROC curves constructed by plotting the sensitivity (true positive rate) versus the1-specificity, or false positive rate, for varying probability cutoff level. visualization of the relationship between sensitivity and specificity of a test. Area under the ROC curves (AUC) measures the probability of the classifier to correctly classify events and nonevents.  Patient Data Unv. Hospitals Leuven 1994~ records, 25 features 32% malignant Univariate Analysis Preprocessing Multivariate Analysis PCA, Factor analysis Stepwise logistic regression Model Building Bayesian LS-SVM + sparse approxi. Bayesian MLP Model Evaluation ROC analysis: AUC Cross validation (temporal, random) Descriptive statistics Input Variable Selection Data Exploration Model Development Procedure of developing models to predict the malignancy of ovarian tumors Goal: find a model  With High sensitivity for malignancy and low false positive rate.  Providing probability of malignancy for individual. Bayesian LS-SVM (RBF, Linear) Forward Selection (Max. Evidence) 3. Methods

4. Bayesian MLPs and Bayesian LS-SVMs for classification LS-SVM Classifier (VanGestel,Suykens 2002) Computing posterior class probabilities solved in dual space Model evidence Bayesian Evidence Framework Inferences are divided into distinct levels. MLP Classifiers (Mackay 1992)

Computing posterior class probabilities for minimum risk decision making Incorporate the different misclassification costs into the class priors: e.g. Set the adjusted prior probability for malignant and benign class to: 2/3 and 1/3. 5. Experimental results RMI: risk of malignancy index = score morph × score meno × CA125  Training set : data from the first treated 265 patients  Test set : data from the latest treated 160 patients Performance from Temporal validation ROC curve on test set Performance on Test set Input variable selection The forward selection procedure tries to maximize the model evidence of LS-SVM given a certain type of kernel 10 variables were selected using RBF kernels. l_ca125, pap, sol, colsc3, bilat, meno, asc, shadows, colsc4, irreg

 The forward selection procedure which tries to maximize the evidence of LS-SVM model is able to identify the important variables.  The performance of LS-SVMs and MLPs are comparable.  Both models have the potential to give reliable preoperative prediction of malignancy of ovarian tumors.  A larger scale validation is needed. References 1. C. Lu, T. Van Gestel, et al. Preoperative prediction of malignancy of ovarian tumors using Least Squares Support Vector Machines (2002), submitted paper. 2. D. Timmerman, H. Verrelst, et al., Artificial neural network models for the preoperative discrimination between malignant and benign adnexal masses. Ultrasound Obstet Gynecol (1999). 3. J.A.K. Suykens, J. Vandewalle, Least Squares support vector machine classifiers, Neural Processing Letters (1999), 9(3). 4. T. Van Gestel, J.A.K. Suykens, et al., Bayesian framework for least squares support vector machine classifiers, Gaussian process and kernel fisher discriminant analysis, Neural Computation (2002), 15(5). 5. D.J.C. MacKay, The evidence framework applied to classification networks, Neural Computation (1992), 4(5). Performance from randomized cross-validation (30 runs)  randomly separating training set (n=265) and test set (n=160)  Stratified, #malignant : #benign ~ 2:1 for each training and test set.  Repeat 30 times Averaged Performance on 30 runs of validations 6. Conclusions