Recuperação de Informação B Cap. 02: Modeling (Latent Semantic Indexing & Neural Network Model) 2.7.2, September 27, 1999

Latent Semantic Indexing n Classic IR might lead to poor retrieval due to: u unrelated documents might be included in the answer set u relevant documents that do not contain at least one index term are not retrieved u Reasoning: retrieval based on index terms is vague and noisy n The user information need is more related to concepts and ideas than to index terms n A document that shares concepts with another document known to be relevant might be of interest

Latent Semantic Indexing n The key idea is to map documents and queries into a lower dimensional space (i.e., composed of higher level concepts which are in fewer number than the index terms) n Retrieval in this reduced concept space might be superior to retrieval in the space of index terms

Latent Semantic Indexing n Definitions u Let t be the total number of index terms u Let N be the number of documents u Let (Mij) be a term-document matrix with t rows and N columns u To each element of this matrix is assigned a weight wij associated with the pair [ki,dj] u The weight wij can be based on a tf-idf weighting scheme

Latent Semantic Indexing n The matrix (Mij) can be decomposed into 3 matrices (singular value decomposition) as follows: u (Mij) = (K) (S) (D) t u (K) is the matrix of eigenvectors derived from (M)(M) t u (D) t is the matrix of eigenvectors derived from (M) t (M) u (S) is an r x r diagonal matrix of singular values where F r = min(t,N) that is, the rank of (Mij)

Computing an Example n Let (Mij) be given by the matrix u Compute the matrices (K), (S), and (D) t

Latent Semantic Indexing n In the matrix (S), select only the s largest singular values n Keep the corresponding columns in (K) and (D) t n The resultant matrix is called (M) s and is given by u (M)s = (K) s (S) s (D) t u where s, s < r, is the dimensionality of the concept space n The parameter s should be u large enough to allow fitting the characteristics of the data u small enough to filter out the non-relevant representational details s

Latent Ranking n The user query can be modelled as a pseudo- document in the original (M) matrix n Assume the query is modelled as the document numbered 0 in the (M) matrix n The matrix (M) t (M) s quantifies the relantionship between any two documents in the reduced concept space n The first row of this matrix provides the rank of all the documents with regard to the user query (represented as the document numbered 0) s

Conclusions n Latent semantic indexing provides an interesting conceptualization of the IR problem n It allows reducing the complexity of the underline representational framework which might be explored, for instance, with the purpose of interfacing with the user

Neural Network Model n Classic IR: u Terms are used to index documents and queries u Retrieval is based on index term matching n Motivation: u Neural networks are known to be good pattern matchers

Neural Network Model n Neural Networks: u The human brain is composed of billions of neurons u Each neuron can be viewed as a small processing unit u A neuron is stimulated by input signals and emits output signals in reaction u A chain reaction of propagating signals is called a spread activation process u As a result of spread activation, the brain might command the body to take physical reactions

Neural Network Model n A neural network is an oversimplified representation of the neuron interconnections in the human brain: u nodes are processing units u edges are synaptic connections u the strength of a propagating signal is modelled by a weight assigned to each edge u the state of a node is defined by its activation level u depending on its activation level, a node might issue an output signal

Neural Network for IR: n From the work by Wilkinson & Hingston, SIGIR’91 Document Terms Query Terms Documents kaka kbkb kckc kaka kbkb kckc k1k1 ktkt d1d1 djdj d j+1 dNdN

Neural Network for IR n Three layers network n Signals propagate across the network n First level of propagation: u Query terms issue the first signals u These signals propagate accross the network to reach the document nodes n Second level of propagation: u Document nodes might themselves generate new signals which affect the document term nodes u Document term nodes might respond with new signals of their own

Quantifying Signal Propagation n Normalize signal strength (MAX = 1) n Query terms emit initial signal equal to 1 n Weight associated with an edge from a query term node ki to a document term node ki: u Wiq u Wiq = wiq sqrt (  i wiq ) n Weight associated with an edge from a document term node ki to a document node dj: u Wij u Wij = wij sqrt (  i wij ) 2 2

Quantifying Signal Propagation WiqWij n After the first level of signal propagation, the activation level of a document node dj is given by:  i Wiq Wij =  i wiq wij sqrt (  i wiq ) * sqrt (  i wij ) F which is exactly the ranking of the Vector model n New signals might be exchanged among document term nodes and document nodes in a process analogous to a feedback cycle n A minimum threshold should be enforced to avoid spurious signal generation 22

Conclusions n Model provides an interesting formulation of the IR problem n Model has not been tested extensively n It is not clear the improvements that the model might provide