Rey-Long Liu Dept. of Medical Informatics Tzu Chi University Taiwan Identification of Biomedical Articles with Highly Related Core Contents Rey-Long Liu Dept. of Medical Informatics Tzu Chi University Taiwan

Outline Background Problem definition The proposed technique: CCSE (Core Content Similarity Estimation) Empirical evaluation Conclusion

Background

Core Contents of Biomedical Articles Core contents of a scholarly article a are the textual contents about Research goal of a Research background of a Research conclusion of a

Similarity Estimation for the Core Contents Goal: retrieval of highly related articles Mining & analysis of highly related evidence A typical goal of existing search engines Challenge: recognition of the core contents Core content of an article a may be briefly expressed in the title and scattered in the abstract

Selected by biomedical experts for <erythropoietin, anemia>  They are highly related to each other Recommended by PubMed, but NOT highly related to <erythropoietin, anemia> 6

Problem Definition

Goal Developing a technique CCSE (Core Content Similarity Estimation) Given: titles and abstracts of two articles a1 and a2 Output: core content similarity between a1 and a2

Contributions CCSE works on titles and abstracts only, which are publicly available CCSE improves inter-article similarity estimation by considering the core contents of the articles

Related Work Inter-article similarity based on citation links Example I: out-link citations (by bibliographic coupling, BC) Example II: in-link citations (by co-citation, CC) Weakness: The citation links are often not available on the Internet (many articles even have no in-link citations)

Related Work (cont.) Inter-article similarity based on textual contents Working on publicly available parts Titles and abstracts Considering weights of terms (e.g., TFIDF weight) Weakness: Did not consider the core content similarity (due to the difficulty in recognizing the core contents)

The Proposed Technique: CCSE

Main Ideas How are goal terms of a1 related to the goal of a2? Title of a1 Title of a2 Abstract of a1 Abstract of a2 How are background and conclusion terms of a1 related to the background and conclusion of a2, respectively?

Main Ideas (cont.) Three kinds of relatedness of a term t the core content of an article a, Rgoal: Relatedness to goal Rback: Relatedness to background Rconc: Relatedness to conclusion Rgoal, Rback, and Rconc are estimated based on the positions of t in the title and the abstract of a

Step 1/2: Estimation of Rgoal, Rback, and Rconc based on positions of the term:

Step 2/2: Estimating inter-article similarity between two articles a1 and a2  Any mismatch between the core contents will significantly reduce the inter-article similarity  Similarity between a1 and a2 is based on goal match, background match, and conclusion match between a1 and a2

Min  Matchgoal(a1, a2) is based on how terms in the title of a1 is related to the goals of a1 and a2

Min  Matchback(a1, a2) is based on how terms in the abstract of a1 is related to the backgrounds of a1 and a2

Min  Matchconc(a1, a2) is based on how terms in the abstract of a1 is related to the conclusions of a1 and a2

Interesting Features of CCSE Inter-article similarity is composed of three parts Goal similarity Background similarity Conclusion similarity These similarities are estimated based on the positions of the terms appearing in the title and the abstract of the article Any mismatch between the core contents will significantly reduce the inter-article similarity

Empirical Evaluation

The data Two sets of articles Highly related biomedical articles: For each gene-disease pair <g,d>, collect the biomedical articles that biomedical experts selected to annotate the pair (noted by DisGeNET) Near-miss biomedical articles (Non-highly related articles): For each gene-disease pair <g,d>, collect articles using two queries: “g NOT d” and “d NOT g”

Data statistics 53 gene-disease pairs 9,875 articles, including 53 targets + 9,822 candidates 435,786 out-link references

The Baseline Systems (1) Link-based inter-article similarity Bibliographic coupling (BC) (2) Text-based inter-article similarity BM25 (one of the best in the biomedical domain) (3) Biomedical search engine PubMed (popular and one of the best in the biomedical domain)

Evaluation Criteria MAP (Mean Average Precision) If a system can rank higher those articles that are highly related to r, average precision (AvgP) for the gene-disease pair will be higher MAP is simply the average of the AvgP values for all gene-disease pairs

Average P@X If those articles that are highly related to r, are ranked at top-X position, P@X for the gene-disease pair will be higher Average P@X is simply the average of the P@X values for all gene-disease pairs

Result CCSE performs significantly better than BC and BM25

CCSE performs better than PubMed in all evaluation criteria:

Conclusion

Our Motivation: We develop CCSE that The idea of CCSE can be Core contents of scholarly articles are essential for retrieval of highly related scientific evidence, BUT The core contents are scattered in titles and abstracts of articles We develop CCSE that Estimates inter-article similarity based on the similarities in goals, backgrounds, and conclusions of two articles The idea of CCSE can be Incorporated into search engines to properly retrieve highly related scholarly articles