Recuperação de Informação B Cap. 02: Modeling (Set Theoretic Models) 2.6 September 08, 1999

Set Theoretic Models n The Boolean model imposes a binary criterion for deciding relevance n The question of how to extend the Boolean model to accomodate partial matching and a ranking has attracted considerable attention in the past n We discuss now two set theoretic models for this: u Fuzzy Set Model u Extended Boolean Model

Fuzzy Set Model n Queries and docs represented by sets of index terms: matching is approximate from the start n This vagueness can be modeled using a fuzzy framework, as follows: u with each term is associated a fuzzy set u each doc has a degree of membership in this fuzzy set n This interpretation provides the foundation for many models for IR based on fuzzy theory n In here, we discuss the model proposed by Ogawa, Morita, and Kobayashi (1991)

Fuzzy Set Theory n Framework for representing classes whose boundaries are not well defined n Key idea is to introduce the notion of a degree of membership associated with the elements of a set n This degree of membership varies from 0 to 1 and allows modeling the notion of marginal membership n Thus, membership is now a gradual notion, contrary to the crispy notion enforced by classic Boolean logic

Fuzzy Set Theory n Definition u A fuzzy subset A of U is characterized by a membership function  (A,u) : U  [0,1] which associates with each element u of U a number  (u) in the interval [0,1] n Definition u Let A and B be two fuzzy subsets of U. Also, let ¬A be the complement of A. Then, F  (¬A,u) = 1 -  (A,u) F  (A  B,u) = max(  (A,u),  (B,u)) F  (A  B,u) = min(  (A,u),  (B,u))

Fuzzy Information Retrieval n Fuzzy sets are modeled based on a thesaurus n This thesaurus is built as follows: u Let vec(c) be a term-term correlation matrix u Let c(i,l) be a normalized correlation factor for (ki,kl): c(i,l) = n(i,l) ni + nl - n(i,l) ni: number of docs which contain ki nl: number of docs which contain kl n(i,l): number of docs which contain both ki and kl n We now have the notion of proximity among index terms.

Fuzzy Information Retrieval n The correlation factor c(i,l) can be used to define fuzzy set membership for a document dj as follows:  (i,j) = 1 -  (1 - c(i,l)) ki  dj  (i,j) : membership of doc dj in fuzzy subset associated with ki n The above expression computes an algebraic sum over all terms in the doc dj n A doc dj belongs to the fuzzy set for ki, if its own terms are associated with ki

Fuzzy Information Retrieval n  (i,j) = 1 -  (1 - c(i,l)) ki  dj  (i,j) : membership of doc dj in fuzzy subset associated with ki n If doc dj contains a term kl which is closely related to ki, we have u c(i,l) ~ 1 u  (i,j) ~ 1 u index ki is a good fuzzy index for doc

Fuzzy IR: An Example n q = ka  (kb   kc) n vec(qdnf) = (1,1,1) + (1,1,0) + (1,0,0) = vec(cc1) + vec(cc2) + vec(cc3) n  (q,dj) =  (cc1+cc2+cc3,j) = 1 - (1 -  (a,j)  (b,j)  (c,j)) * (1 -  (a,j)  (b,j) (1-  (c,j))) * (1 -  (a,j) (1-  (b,j)) (1-  (c,j))) cc1 cc3 cc2 KaKb Kc

Fuzzy Information Retrieval n Fuzzy IR models have been discussed mainly in the literature associated with fuzzy theory n Experiments with standard test collections are not available n Difficult to compare at this time

Extended Boolean Model n Booelan retrieval is simple and elegant n But, no ranking is provided n How to extend the model? u interpret conjunctions and disjunctions in terms of Euclidean distances