Adding Semantics to Information Retrieval By Kedar Bellare 20 th April 2003

Motivation Current IR techniques term-based Semantics of document and query not considered Problems like polysemy and synonymy Lot of advances in NLP and Statistical Modeling of Semantics Is Semantic IR really required?

Organization Traditional IR Statistics for Semantics – Latent Semantic Indexing Semantic Resources for Semantics – Use of Semantic Nets, Conceptual Graphs, WordNet etc. in IR. Conclusion

Information Retrieval An information retrieval system does not inform the user on the subject of his inquiry. It merely informs on the existence (or non- existence) and whereabouts of documents relating to his request.

A Typical IR System

Current IR Preprocessing of Documents Inverted Index Removing stopwords and Stemming Representation of Documents Vector Space Model – TF and IDF Document Clustering Improvements to the above Better weighting of Document Vectors Link analysis – PageRank and Anchor Text

Latent Semantic Indexing Problems with Traditional Approaches Synonymy – Automobile and Car Polysemy – Jaguar means both a Car and Animal LSI – Linear Algebra for capturing “Latent Semantics” of documents Method of dimensionality reduction

LSI Compares document vectors in Latent Semantic Space Two documents can have high similarity value even if no terms shared Attempts to remove minor differences in terminology during indexing Truncated SVD – used for construction of Latent Semantic Space

Singular Value Decomposition Given a term-document matrix A t x d converts it into product of three matrices T t x r, S r x r and D d x r such that A = T S D T T and D are orthogonal, S is diagonal and r is rank of A Reduced space corresponds to axes of greatest variation

What LSI does? Uses truncated SVD Instead of r – dimensional space uses a factor k Ā t x d = T t x k S k x k D T d x k Truncated SVD – captures underlying structure in association of terms and documents

Using the SVD model Comparison of terms – entries of the matrix T S 2 T T Comparison of documents – entries of the matrix D S 2 D T Comparison of term and document – entries of the matrix TSD T Query in SVD model – q’ = q T T S -1

Example of LSI

Why LSI works? Although lot of empirical evidence no concrete proof of why LSI works No major degradation – Theorem of Eckart and Young States that the distance of two matrices is minimum Still does not explain improvements in recall and precision

Why LSI works? (contd.) Papadimitriou et. al. Assumes documents generated from set of topics with disjoint vocabularies If term-document matrix A is perturbed, they prove that LSI recovers topic information and removes the noise Kontostathis et. al. Essentially claims that LSI’s ability to trace term co-occurrences is what helps in improved recall

Advantages & Disadvantages Advantages Synonymy Term Dependence Disadvantages Storage Efficiency

Semantic Resources Semantic Nets - E.g. John gave Mary the book Applied in UNL – Eg. Only a few farmers could use information technology in early 1990s

Semantic Resources (contd.) Conceptual Graphs – E.g. A bird is singing in a Sycamore tree Conceptual Dependency – E.g. I gave the man a book Lexical Resources – WordNet

Applications of Semantic Resources in IR UNL Used in improving document vectors Conceptual Graphs Graph matching of query and document CDs FERRET – Comparison of CD patterns WordNet Query Expansion using WordNet

Conclusion Various things need to be considered before applying to Web Storage Efficiency Knowledge Content of Query Clearly, semantic method needed for eliminating synonymy and polysemy Currently, traditional models with minor hacks serve the purpose However, in conclusion : Statistical or Conceptual or combination of both to model Document Semantics is definitely required

References [1] M. W. Berry, S. T. Dumais, and G. W. O’Brien. Using linear algebra for intelligent information retrieval. SIAM Review, 37(4), pages 573–595, [2] S. Chakrabarti. Mining the Web - Discovering Knowledge from Hypertext Data. Morgan Kau.mann Publishers, San Francisco, [3] S. Deerwester, S. T. Dumais, T. K. Landauer, G. W. Furnas, and R. A. Harshman. Indexing by Latent Semantic Analysis. Journal of the Society for Information Science 41 (6), pages 391–407, [4] A. Kontostathis and W. M. Pottenger. A mathematical view of Latent Semantic Indexing: Tracing Term Co-occurences. Technical report, Lehigh University, [5] R. Mandala, T. Takenobu, and T. Hozumi. The use of WordNet in Information Retrieval. In COLING/ACL Workshop on the Usage of WordNet in Natural Language Processing Systems, pages 31–37, 1998.

References (contd.) [6] M. L. Mauldin. Retrieval performance in FERRET: a conceptual information retrieval system. In Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, pages 347–355. ACM Press, [7] G. A. Miller, R. Beckwith, C. Fellbaum, D. Gross, and K. J.Miller. Introduction to WordNet: an on-line lexical database. International Journal of Lexicography 3 (4), pages 235 – 244, [8] M. Montes-y-Gomez, A. Lopez, and A. F. Gelbukh. Information retrieval with Conceptual Graph matching. In Database and Expert Systems Applications, pages 312–321, [9] C. H. Papadimitriou, H. Tamaki, P. Raghavan, and S. Vempala. Latent Semantic Indexing: A probabilistic analysis. pages 159–168, [10] E. Rich and K. Knight. Artificial Intelligence. Tata McGraw-Hill Publishers, New Delhi, 2002.

References (contd.) [11] G. Salton, A. Wong, and C. S. Yang. A vector space model for automatic indexing. Communications of the ACM, 18(11):613– 620, [12] C. Shah, B. Chowdhary, and P. Bhattacharyya. Constructing better Document Vectors using Universal Networking Language (UNL). In Proceedings of International Conference on Knowledge-Based Computer Systems (KBCS) NCST, Navi Mumbai, India, [13] H. Uchida, M. Zhu, and S. T. Della. UNL : A gift for a millenium. Technical report, The United Nations University, [14] C. J. van Rijsbergen. Information Retrieval. Butterworths, London, 1979.