1. Datamining MEDLINE for Topics and Trends in Dental and Craniofacial Research William C. Bartling, D.D.S. NIDCR/NLM Fellow in Dental Informatics Center for Biomedical Informatics University of Pittsburgh Titus K. L. Schleyer, D.M.D., Ph.D. Director, Center for Dental Informatics University of Pittsburgh School of Dental Medicine 1 of 25

2. Overview Goals of project Goals of project Steps involved in the project Steps involved in the project Present status of the project Present status of the project Summary Summary 2 of 25

3. Goal of the project To perform an accurate and thorough content analysis of the published dental and craniofacial research literature indexed in MEDLINE To perform an accurate and thorough content analysis of the published dental and craniofacial research literature indexed in MEDLINE 3 of 25

4. Outline of project 1. Develop a search strategy: –retrieve all references to dental and craniofacial journal articles indexed in MEDLINE 2. Determine the characteristics of a dental or craniofacial “research” article. 3. Use computer programs to “learn” these characteristics in order to facilitate retrieval. 4. Use computerized methods to identify trends in dental and craniofacial research. 4 of 25

5. 1. Search strategy MEDLINE was chosen as the database. MEDLINE was chosen as the database. MeSH was searched manually for dental and craniofacial descriptors. MeSH was searched manually for dental and craniofacial descriptors. Decision to keep or discard term Decision to keep or discard term Text word searches to determine completeness Text word searches to determine completeness Search limited to : Search limited to : –English language –Publication type –Abstract present –1966-October, 2002 5 of 25

6. Results of search ~450,000 English language articles in: ~450,000 English language articles in: –DENTISTRY –STOMATOGNATHIC SYSTEM (not PHARYNX) –STOMATOGNATHIC DISEASES (not PHARYNGEAL DISEASES) ~61,000 articles indexed with dental MeSH terms not in above set ~61,000 articles indexed with dental MeSH terms not in above set ~134,000 articles remaining after limiting to journal articles containing abstracts ~134,000 articles remaining after limiting to journal articles containing abstracts 6 of 25

7. 2. What is a dental research article? 1000 abstracts randomly chosen 1000 abstracts randomly chosen – 5 groups of 200 abstracts each 16 expert judges 16 expert judges 3 judges assigned to each group (1 group: n = 4) 3 judges assigned to each group (1 group: n = 4) Judges categorize each article as: Judges categorize each article as: –Dental or craniofacial research –Dental or craniofacial, non-research –Non-dental –Not sure Web interface for judging- PHP with mySQL Web interface for judging- PHP with mySQL Web 7 of 25

8. 8 of 25

9. 9 of 25

10. Expert judging results Acceptable reliability in each group (  > 0.70) Acceptable reliability in each group (  > 0.70) –0.70, 0.75, 0.85, 0.77, 0.76 Majority rating determined abstract category. Majority rating determined abstract category. Dental research 59160% Dental non-research 11512% Non-dental12913% Not sure 3 0% 0% No majority 15215% TOTAL990100% 10 of 25

11. Title and MeSH findings RatingN Unique journal titles Unique major MeSH % in “Dental” Journals Dental Research 5912501027 (2092 total) 62% Dental Non- Research11578224 (321 total) 68% Non-dental129120411 (478 total) 5% 11 of 25

12. Dental research top 10 journal titles (n = 591) J Periodontol254.2% J Periodontol254.2% J Dent Res 203.4% J Dent Res 203.4% Arch Oral Biol 183.0% Arch Oral Biol 183.0% J Prosthetic Dent 132.2% J Prosthetic Dent 132.2% J Clin Periodontol 132.2% J Clin Periodontol 132.2% Community Dent Oral Epidemiology 111.9% Community Dent Oral Epidemiology 111.9% Oral Surg Oral Med Oral Pathol 101.7% Oral Surg Oral Med Oral Pathol 101.7% J Oral Rehabil101.7% J Oral Rehabil101.7% Tie: Scand J Dent Res, J Oral Maxillofac Surg, Tie: Scand J Dent Res, J Oral Maxillofac Surg, J Endod, Am J Ortho Dentofacial Orthoped 91.5% 12 of 25

13. Dental non-research top 10 journal titles (n = 115) J Prosthet Dent119.6% J Prosthet Dent119.6% Br Dent J 54.3% Br Dent J 54.3% J Dent Educ 43.5% J Dent Educ 43.5% J Dent Child 43.5% J Dent Child 43.5% J Am Coll Dent 32.6% J Am Coll Dent 32.6% Arch Otolaryngol 32.6% Arch Otolaryngol 32.6% Tie: 13 journals 21.7% Tie: 13 journals 21.7% 13 of 25

14. Non-dental top 10 journal titles (n = 129) J Biol Chem43.1% J Biol Chem43.1% J Comp Neurol21.6% J Comp Neurol21.6% J Cell Biol21.6% J Cell Biol21.6% Biochim Biophys Acta21.6% Biochim Biophys Acta21.6% Biochemistry21.6% Biochemistry21.6% Anat Rec21.6% Anat Rec21.6% Acta Otolaryngol(Stockh)21.6% Acta Otolaryngol(Stockh)21.6% 113 other titles10.8% 113 other titles10.8% 26 of these titles also occur in “dental research” set. 26 of these titles also occur in “dental research” set. 14 of 25

15. Journals containing both “dental research” AND “non-dental” J Biol Chem J Biol Chem Biochim Biophys Acta Biochim Biophys Acta Anat Rec Anat Rec Physiol Behav Physiol Behav Oral Surg Oral Med Oral Pathol Oral Surg Oral Med Oral Pathol Laryngoscope Laryngoscope J Oral Pathol Med J Oral Pathol Med J Oral Maxillofac Surg J Oral Maxillofac Surg J Invest Dermatol J Invest Dermatol J Infect Dis J Infect Dis J Dent Educ J Dent Educ J Bone Miner Res J Bone Miner Res J Bacteriol J Bacteriol Int J Biochem Int J Biochem Infect Immun Infect Immun FEBS Lett FEBS Lett Eur J Pharmacol Eur J Pharmacol Eur J Cell Biol Eur J Cell Biol Carbohydr Res Carbohydr Res Cancer Res Cancer Res Arzneimittelforschung Arzneimittelforschung Arch Dis Child Arch Dis Child Anat Anz Anat Anz Am J Epidemiol Am J Epidemiol Am J Anat Am J Anat Acta Pathol Microbiol Immunol Scand Acta Pathol Microbiol Immunol Scand Can we limit our search by journal type? 15 of 25

16. Top dental research major MeSH (n = 591) Dental Caries Dental Caries Mouth Neoplasms Mouth Neoplasms Mouth Mucosa Mouth Mucosa Gingiva Gingiva Periodontal Diseases Periodontal Diseases Periodontitis Periodontitis Mandible Mandible Dental Implants Dental Implants Carcinoma, Squamous Cell Carcinoma, Squamous Cell Submandibular Gland Submandibular Gland 274.6% 274.6% 223.7% 223.7% 193.2% 193.2% 183.0% 183.0% 172.9% 172.9% 152.5% 152.5% 142.4% 142.4% 16 of 25

17. Dental non-research major MeSH n = 115 Dental Education Dental Education Denture Design Denture Design Mandible Mandible Maxilla Maxilla Dentistry Dentistry Surgical Flaps Surgical Flaps Osteotomy Osteotomy Denture, Complete Denture, Complete Dentists Dentists Dental Restoration, Permanent Dental Restoration, Permanent 76.1% 76.1% 65.2% 65.2% 54.3% 54.3% 43.5% 43.5% 17 of 25

18. Non-dental major MeSH n = 129 Submandibular Gland Submandibular Gland Skin Skin Epidermal Growth Factor Epidermal Growth Factor Adenylate Cyclase Adenylate Cyclase Salivary Glands Salivary Glands Hip Prosthesis Hip Prosthesis Fluorides Fluorides Chromosomes Chromosomes Chlorides Chlorides Cementation Cementation 43.1% 43.1% 32.3% 32.3% 18 of 25

19. 3. Use computers to extract dental research articles IPS-Identify Patient Sets (Dr. Greg Cooper et al, University of Pittsburgh) IPS-Identify Patient Sets (Dr. Greg Cooper et al, University of Pittsburgh) Uses naïve Bayes method of “text classification.” Uses naïve Bayes method of “text classification.” Training set (n = 693) to train system Training set (n = 693) to train system Test set (n = 297) to test system Test set (n = 297) to test system Calculate success of retrieval in test set using model created from training set. Calculate success of retrieval in test set using model created from training set. 19 of 25

20. IPS creates a document vector for each document or set of documents Document i Word 3 p 3 Word n p n Word 1 p 1 Word 2 p 2 20 of 25

21. IPS results Title and abstract only Titles&Abstracts Recall or Sensitivity Specificity Precision or PPV Training set (n = 693) 0.630.790.82 Test set (n = 297) 0.640.620.71 21 of 25

22. Additional retrieval methods JDI-Journal Descriptor Indexing(NLM) JDI-Journal Descriptor Indexing(NLM) –Takes free text of title and abstract and assigns journal “type” MTI-Medical Text Indexer(NLM) MTI-Medical Text Indexer(NLM) –Assigns MeSH descriptors based on free text Will compare performance of these methods with IPS Will compare performance of these methods with IPS 22 of 25

23. 4. Identify trends in dental research Use the appropriate method to successfully retrieve all dental research articles from MEDLINE. Use the appropriate method to successfully retrieve all dental research articles from MEDLINE. Using information visualization and/or representation techniques to Using information visualization and/or representation techniques to –Analyze content of dental research literature –Identify trends in dental research –Identify opportunities in dental research 23 of 25

24. Summary A comprehensive content analysis of the dental and craniofacial research literature has not been done. A comprehensive content analysis of the dental and craniofacial research literature has not been done. Computerized methods may help to retrieve and categorize this literature. Computerized methods may help to retrieve and categorize this literature. Study of the published dental research can help researchers to identify trends and to possibly reveal future research opportunities. Study of the published dental research can help researchers to identify trends and to possibly reveal future research opportunities. 24 of 25

25. Many thanks to the following: Amy Gregg, MLIS-Dental Reference Librarian Falk Library for the Health Sciences University of Pittsburgh Shyam Visweswaran, MD- NLM Fellow in Intelligent Systems Center for Biomedical Informatics University of Pittsburgh All of my expert raters ! This research is supported with a training grant from the National Institute of Dental and Craniofacial Research and the National Library of Medicine 25 of 25