1. Using a Convolutional Neural Network model for Automatic Medical Subject Headings (MeSH) Assignment
Wei Wei, PhD, Zhanglong Ji, PhD, Lucila Ohno-Machado, MD, PhD
University of California, San Diego
Abstract
Assigning Medical Subject Headings (MeSH) is an important step in indexing biomedical research articles in the National Library of Medicine (NLM). Automated MeSH assignment methods have been developed to help NLM indexers, and most of these methods use manually crafted features. In recent years, data-driven features learned using deep learning methods have been proven to capture abundant information, and therefore have been widely used in natural language processing (NLP) research. We studied the distributed representation features generated from a deep learning method, Convolutional Neural Networks (CNN), in a MeSH assignment task, and found that CNNs can result in assignment performance comparable to that of methods based on manually crafted features. These data-driven features can also improve model performance without requiring additional training data.
Figure 1. Using CNN features and additional features to rank MeSH term candidates. Although only one convolutional filter is illustrated, multiple filters were used in experiments.
The CNN and additional features together achieved good MAP on NLM2007, comparable to using the 11 manual features in the baseline model (details in Table 1). The neighborhood feature has a dominant impact on the performance (Table 1, row 2), while the other features were valuable supplements.
Results
Introduction
Medical Subject Headings (MeSH) is a controlled vocabulary thesaurus used primarily for indexing biomedical publications. Assigning MeSH terms enables efficient retrieval of biomedical publications via PubMed. Most assignment methods use manually extracted features, including the state-of-art model1. Inspired by the performance of the Convolutional Neural Network (CNN) model for feature extraction in text mining research2, we studied the contribution of automatically generated features from CNN for the MeSH term assignment task.
Features
MAP
Baseline (11 manually extracted features)
0.6263
Neighborhood
0.6023
CNN
0.335
Neighborhood + CNN
0.602
Neighborhood + CNN + Overlap + Translation
0.646
Table 1. Performance of different combinations.
Methods
The model built with CNN and the three additional features helped improve MAP on the NLM2007 dataset. Neither the CNN features nor the additional features require extra training data.
Conclusions
A CNN model2 generated distributed representations (i.e., CNN features) of terms from abstracts and MeSH term candidates, followed by a learning-to-rank algorithm to score the relevance of the candidates3 (Figure 1). The model by Huang et al.4 served as the baseline. The baseline model includes three features (i.e., additional features) that do not require extra training data:
1. Neighborhood feature: the number of similar articles in
which a MeSH term candidate appears,
2. Overlap feature: number of unigrams/bigrams overlapping
between the MeSH term candidate and the abstract, and
3. Translation feature: the probability of translating the abstract
into the MeSH term candidate.
We analyzed the contribution of CNN features alone, together with the neighborhood feature, and with all the three features. The performance was evaluated on Dataset NLM2007, a National Library of Medicine (NLM) benchmarking dataset4 containing 200 MEDLINE records.
The performance was measured by Mean Average Precision (MAP), which is the average precision values at ranks of relevant MeSH terms.
The scripts and the configuration of CNN are available from
References
1. Liu K, Peng S, Wu J, Zhai C, Mamitsuka H, Zhu S. MeSHLabeler: improving the accuracy of large-scale MeSH indexing by integrating diverse evidence. Bioinformatics. 2015;31(12):i
2. Kim Y. Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing p. 1746–51.
3. Severyn A, Moschitti A. Learning to rank short text pairs with convolutional deep neural networks. In: Proceedings of the 38th ACM SIGIR Conference p
4. Huang M, Névéol A, Lu Z. Recommending MeSH terms for annotating biomedical articles. J Am Med Inform Assoc. 2011;18(5):660–7.
This study was supported in part by bioCADDIE grant NIH/BD2K - U24AI to the University of California, San Diego