Jen-Tzung Chien, Meng-Sung Wu Minimum Rank Error Language Modeling

Introduction Language model for information retrieval Minimum rank error model Experiments Conclusion Outline

the language model is useful for investigating the linguistic regularities in queries and documents for information retrieval But the accuracy of classifying queries into the relevant documents is not concerned with the ranks of the retrieved documents MCE training is also used in IR. In the MCE procedure, the expected loss function is minimized with probabilistic descent algorithm for optimal Bayes risk Introduction

With MCE, the rate of misclassification is reduced. But rank result is still not consist with the performance measure, i.e. AP The minimum rank error (MRE) language model is established by a gradient descent algorithm to obtain discriminative retrieval for training queries with minimum expected rank error loss Introduction

Language model for information retrieval

the document terms are often too few to train reliable ML model. Many words are unseen in the document, leading to zero probabilities in many n-gram events the smoothed language model is obtained by linear interpolation of the document and background models Language model for information retrieval

MCE is a training method based on Bayes decision theory. This method can reduce misclassification by minimize the expect loss with three step procedure First, a misclassification measure is defined Second, the misclassification measure is normalized as the classification error loss function ranging between 0 and 1 by the sigmoid function given as follows Minimum Classification Error Model

Receiver Operating Characteristic (ROC) is one kind of measure which consider the true positive rate and false positive rate; Area Under ROC Curve(AUC) gives a value for the ROC curve Information Retrieval Measures

Average Precision (AP) Versus Rank Error Minimum Rank Error (MRE) Model Implementation and Interpretation Minimum rank error model

The information retrieval model can be estimated by optimizing the AP, but the minimization of the expected AP loss function is mathematically intractable So we develop the rank error loss function instead of the classification error loss Average Precision (AP) Versus Rank Error

Minimum Rank Error (MRE) Model

The rank error loss function is calculated by substituting the misranking measure into sigmoid function. And the expect rank error is calculated over the entire training set including all query and their relevant documents Minimum Rank Error (MRE) Model

The document model is iteratively updated by the descent algorithm Considering a logarithm bigram in document model, the differentials are calculated by Minimum Rank Error (MRE) Model

The figure below shows the procedure of MRE language model training for information retrieval Implementation and Interpretation

MRE and MCE are derived as the discriminative learning algorithms from the same Bayes decision theory, but they are different by two aspects In performance metrics –MRE minimizes the Bayes rank risk based on the rank error loss function –MCE minimizes the Bayes risk due to classification errors In use of training data –The MRE model uses queries and their corresponding document lists as training samples –MCE considers all irrelevant documents in a rank list Implementation and Interpretation

Experiments

Most classification systems are based on minimization of classification errors, and thus do not reflect the ranking performance of retrieval systems This paper focuses on the ranking problem, and presents a new discriminative retrieval model. The experiment results also shows MRE retrieves more relevant documents with high ranks than MCE Conclusion