Evaluation of N-grams Conflation Approach in Text-based Information Retrieval Serge Kosinov University of Alberta, Computing Science Department, Edmonton, AB, Canada

N-gram Conflation Method Goal study a conflation method based on n-gram approach with some enhancements and evaluate its performance in textual information retrieval What is conflation good for? matching non-identical words that refer to the same principle concept Why is it important? Avoid problems of strong dependence of retrieval results on the exact wording of a user's query  account for richness and redundancy of the natural language

N-grams Method: Basic Idea * Subdivide words into N-grams - set of overlapping substrings of length N Example:N=2: (radio)  ( ra - ad - di - io ) N=3: (radio)  ( rad - adi - dio ) * Treat as similar and group together words with identical N-gram structure

N-grams Method: Basic Idea (Continued)

Experiment Implementation * Pre-process text collections (remove stop words, punctuation, special characters, etc.) * Find the set of unique terms and compute their similarity matrix * Cluster this data and compute IDF-like correction multipliers for each N-gram * Process queries by replacing the terms that fit into obtained clusters with cluster ID both in document collections and in queries, then pick the best match via standard vector model representation

Computing Similarity Matrix

* Clustering technique complete link agglomerative clustering (aka HCA) Example: C325 = {computer, computing, computer-based} * IDF-like adjustments of weights w ij - weight of bigram B j in term cluster C i bf ij - frequency of bigram B j in term cluster C i N - number of term clusters n - number of term clusters where bigram B j occurs at least once Clustering Data and Adjusting IDF

Processing Queries: Example Best match : cosine similarity coefficient between document vector (..., C325, C487, Torvalds,... ) and query vector (..., C325, Torvalds,... )

Experimental Results Text collections used: Results: 3 point precision average (at 20, 50, 80 % recall)

Inverse Frequency Weights Effect Association of unseen query terms with clusters: * With IDF-like correction: consolidation  {console} editing  {editor, edition} * Without IDF-like correction : consolidation  {condensation} editing  {accrediting}

Individual Query Analysis Example * Other examples in which N-gram conflation outperforms other methods: criteria-criterion, exchange-interchange, system-subsystem, etc.

Conlusions and Directions for Further Study Advantages of N-gram conflation: * is a language-independent approach * tackles well misprints and orthographical errors * best gain for special form and compound words * enhanced with IDF-like correction performs better than traditional stemming (Porter, etc.) Disadvatages: * clusters have homophone noise * straightforward HCA impractical on large-scale datasets Prospects: * apply the method for more inflected languages * combine N-grams and Porter * enhance clustering routines