Extracting Keyphrases from Books using Language Modeling Approaches Rohini U AOL India R&D, Bangalore India Bangalore Vamshi Ambati Language Technologies Institute Carnegie Mellon University Pittsburgh, USA

Agenda Keyphrase Extraction Keyphrase Extraction Value addition to Digital Libraries Value addition to Digital Libraries Methods of Keyphrase Extraction Methods of Keyphrase Extraction Related Work Related Work Our Solution Our Solution

What are Keyphrases? Keyphrases Keyphrases (Give example) (Give example) Where used? Where used? Cataloguing in Libraries for IR purposes Cataloguing in Libraries for IR purposes Quick Summarization of documents Quick Summarization of documents

Why important to ULIB? Vast growth in digital content Vast growth in digital content More than a Million books! More than a Million books! Short Meta data description – useful to user while reading Short Meta data description – useful to user while reading For further processing of books like summarization, IR etc For further processing of books like summarization, IR etc

How do we extract KPs? Manual entry Manual entry Reliable, high quality outcome Reliable, high quality outcome But, time-consuming, expensive But, time-consuming, expensive Automatic Automatic Fast extraction but less reliable Fast extraction but less reliable No expense at all No expense at all

Automatic techniques for KPE Rule based methods Rule based methods Heuristics (paragraph beginning, headline etc) Heuristics (paragraph beginning, headline etc) Krulwich &Burkey etc Krulwich &Burkey etc Using Linguistic tools Using Linguistic tools Statistical techniques Statistical techniques Term counts and weighting based Methods Term counts and weighting based Methods Learn model from training data Learn model from training data Turney et. al[5], KEA[6], KSpotter[3] etc Turney et. al[5], KEA[6], KSpotter[3] etc

Requirements for a KPE for ULIB Automatic Identification of Keyphrases from chapters of books Automatic Identification of Keyphrases from chapters of books Language independent Language independent Easily adaptable for different domains Easily adaptable for different domains No training data to learn from No training data to learn from Most books in ULIB do not have keywords as part of the metadata Most books in ULIB do not have keywords as part of the metadata

Solution Outline Language Modeling based Language Modeling based Given n-grams Given n-grams Measure Informativeness, Phraseness Measure Informativeness, Phraseness Score n-grams based on the above measures Score n-grams based on the above measures Pick top K phrases as Keyphrases Pick top K phrases as Keyphrases

Extracting Keyphrases from Books Cleaning & Initialization Candidate Keyphrases Extraction Scoring Pruning Extracted Keyphrases Text

Extracting Keyphrases from Books Topics are also used to construct user profiles via explicit specication of interests or automatic analysis of Web pages visited Extracted Keyphrases Cleaning & Initialization Candidate Keyphrases Extraction Scoring Pruning topics construct user profiles explicit specification interests automatic analysis web pages visited

Extracting Keyphrases from Books Topics are also used to construct user proles via explicit specication of interests or automatic analysis of Web pages visited Extracted Keyphrases Cleaning & Initialization Candidate Keyphrases Extraction Scoring Pruning topics construct user profiles explicit specification interests automatic analysis web pages visited {topics construct user, construct user profiles, user profiles explicit, profiles explicit specification, explicit specification interests, specification interests automatic, automatic analysis web, analysis web pages, web pages visited }

Extracting Keyphrases from Books Topics are also used to construct user proles via explicit specication of interests or automatic analysis of Web pages visited Extracted Keyphrases Cleaning & Initialization Candidate Keyphrases Extraction Scoring Pruning topics construct user profiles explicit specification interests automatic analysis web pages visited profiles explicit specication : explicit specication interests : specication interests automatic : user proles explicit : construct user proles : interests automatic analysis : topics construct user : automatic analysis web : web pages visited : analysis web pages :

Scoring Phraseness Phraseness Measures degree to which a given n-gram can be considered a phrase Measures degree to which a given n-gram can be considered a phrase Based on Co-occurrence of words Based on Co-occurrence of words Example.. Example.. Informativeness Informativeness Measures how informative a given n-gram is Measures how informative a given n-gram is There is a, a lot of etc There is a, a lot of etc Comparing co occurrence on a general corpus Vs given text(book) Comparing co occurrence on a general corpus Vs given text(book) Total Score Total Score Phraseness-Score + Informativeness-Score Phraseness-Score + Informativeness-Score

Scoring - Phraseness Computed by measuring distance between unigram model and N-gram model Computed by measuring distance between unigram model and N-gram model Point wise KL-divergence (Takashi et. al 2004) Point wise KL-divergence (Takashi et. al 2004) δ δ w (p||q) = p(w)log(p(w)/q(w)) Phraseness measure Phraseness measure δ δ w (LM fg N || LM fg 1 )

Scoring - Informativeness Computed by measuring distance between n-gram model from given data and n- gram model from general data Computed by measuring distance between n-gram model from given data and n- gram model from general data Point wise KL-divergence (Takashi et. al 2004) Point wise KL-divergence (Takashi et. al 2004) δ δ w (p||q) = p(w)log(p(w)/q(w)) Informativeness measure Informativeness measure δ δ w (LM fg 1 || LM bg 1 )

Extracting Keyphrases from Books Topics are also used to construct user proles via explicit specication of interests or automatic analysis of Web pages visited Extracted Keyphrases Cleaning & Initialization Candidate Keyphrases Extraction Scoring Pruning topics construct user profiles explicit specification interests automatic analysis web pages visited profiles explicit specication : explicit specication interests : specication interests automatic : user proles explicit : construct user proles : interests automatic analysis : topics construct user : automatic analysis web : web pages visited : analysis web pages :

Extracting Keyphrases from Books Topics are also used to construct user proles via explicit specication of interests or automatic analysis of Web pages visited Extracted Keyphrases Cleaning & Initialization Candidate Keyphrases Extraction Scoring Pruning topics construct user profiles explicit specification interests automatic analysis web pages visited proles explicit specication explicit specication interests specication interests automatic user proles explicit construct user proles interests automatic analysis topics construct user automatic analysis web web pages visited analysis web pages

Conclusions and Future Work Discussed benefits of Keyphrases in ULIB context Discussed benefits of Keyphrases in ULIB context Demonstrated the building of a KPE that works for books Demonstrated the building of a KPE that works for books Robust evaluation Robust evaluation Building a test set from books in ULIB for generic robust evaluation of KPE tools Building a test set from books in ULIB for generic robust evaluation of KPE tools Are chapters really independent in a book Are chapters really independent in a book Revisit the assumption Revisit the assumption

Thank you

References Fred J. Damerau. Generating and evaluating domain-oriented multi-word terms from texts. Information Processing and Management, 29(4): , S.T Dumais, J Platt, D. Heckerman, and M. Sahami. Inductive learning algorithms and representations for text categorization. In Proceedings of the 7th international conference on information and knowledge management, page ACM Press, Min Song, Il-Yeol Song, and Xiaohua Hu. Kpspotter: a exible information gain-based keyphrase extraction system. In WIDM '03: Proceedings of the 5th ACM international workshop on Web information and data management, pages 50-53, New York, NY, USA, ACM Press Takashi Tomokiyo and Mathew Hurst. A language modeling approach to keyphrase extraction. In Proceedings of the ACL 2003 workshop on Multiword expressions, pages 33{40, Morristown, NJ, USA, Association for Computational Linguistics P.D. Turney. Learning algorithms for keyphrase extraction. Information Retrieval, 2(4): , I.H. Witten, G.W. Paynter, E. Frank, C. Gutwin, and C.G Nevill-Manning. Kea: Practical automatic keyphrase extraction. In E. A. Fox and N. Rowe, editors, Proceedings of digital libraries 99: The fourth ACM conference on digital libraries, pages ACM Press, Mikio Yamamoto and Kenneth W. Church. Using suffix arrays to compute term frequency and document frequency for all substrings in a corpus. Computational Linguistics, 27(1):1-30, 2001