CSE 635 Multimedia Information Retrieval Chapter 1: Introduction to IR

Motivation IR: representation, storage, organization of, and access to information items Focus is on the user information need User information need: When did the Buffalo Bills last win the Super Bowl? Find all docs containing information on college tennis teams which: (1) are maintained by a USA university and (2) participate in the NCAA tournament. Emphasis is on the retrieval of information (not data)

Information retrieval Motivation Data retrieval which docs contain a set of keywords? Well defined semantics a single erroneous object implies failure! Information retrieval information about a subject or topic deals with unstructured text semantics is frequently loose small errors are tolerated IR system: interpret contents of information items generate a ranking which reflects relevance notion of relevance is most important

Comparison of different information systems Evaluation: Precision versus Recall

IR at the center of the stage Motivation IR at the center of the stage IR in the last 20 years: classification and categorization systems and languages user interfaces and visualization Still, area was seen as of narrow interest Advent of the Web changed this perception once and for all universal repository of knowledge free (low cost) universal access no central editorial board many problems though: IR seen as key to finding the solutions!

The User Task Basic Concepts Retrieval information or data purposeful needle in a haystack problem Browsing glancing around Formula 1 racing; cars, Le Mans, France, tourism Filtering (push rather than pull) Retrieval Browsing Database

Logical view of the documents Basic Concepts Logical view of the documents documents represented by a set of index terms or keywords Document representation viewed as a continuum: logical view of docs might shift Accents spacing Noun groups Automatic or Manual indexing Docs stopwords stemming Structure recognition structure Full text Index terms

The Retrieval Process Indexing Retrieval 4, 10 6, 7 5 8 8 2 Text User Interface 4, 10 user need Text Text Operations 6, 7 logical view logical view Query Operations DB Manager Module Indexing user feedback 5 8 inverted file query Searching Index 8 retrieved docs Text Database Ranking ranked docs 2 Indexing Retrieval

Applications of IR Specialized Domains Information filtering biomedical, legal, patents, intelligence Information filtering Summarization Cross-lingual Retrieval Question-Answering Systems Ask Jeeves Web applications shopbots personal assistant agents Text Mining data mining on unstructured text Multimedia IR images, document images, speech, music

IR Techniques Machine learning clustering, SVM, latent semantic indexing, etc. improving relevance feedback, query processing etc. Natural Language Processing, Computational Linguistics better indexing, query processing incorporating domain knowledge: e.g., synonym dictionaries use of NLP in IR: benefits yet to be shown for large-scale IR Information Extraction Highly focused Natural language processing (NLP) named entity tagging, relationship/event detection Text indexing and compression User interfaces and visualization AI advanced QA systems, inference, etc.

Issues to be Addressed by IR How to improve quality of retrieval Faster indexes and smaller query response times better understanding of user behaviour interactive retrieval visualization techniques