Introduction to IR Research ChengXiang Zhai Department of Computer Science Graduate School of Library & Information Science Institute for Genomic Biology, Statistics University of Illinois, Urbana-Champaign http://www-faculty.cs.uiuc.edu/~czhai, czhai@cs.uiuc.edu 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

What is Research? Research Basic research Applied research Discover new knowledge Seek answers to questions Basic research Goal: Expand man’s knowledge (e.g., which genes control social behavior of honey bees? ) Often driven by curiosity (but not always) High impact examples: relativity theory, DNA, … Applied research Goal: Improve human condition (i.e., improve the wolrd) (e.g., how to cure cancers?) Driven by practical needs High impact examples: computers, transistors, vaccinations, … The boundary is vague; distinction isn’t important 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

Why Research? Funding Curiosity Quality of Life Utility of Applications Advancement of Technology Amount of knowledge Application Development Applied Research Basic Research 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

Where’s IR Research? Funding Quality of Life Utility of Applications Information Science Advancement of Technology Amount of knowledge Computer Science Application Development Applied Research Basic Research 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

Different position benefits from different collaborators Where’s Your Position? Different position benefits from different collaborators Quality of Life Utility of Applications Advancement of Technology Amount of knowledge Application Development Applied Research Basic Research 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

Research Process Identification of the topic (e.g., Web search) Hypothesis formulation (e.g., algorithm X is better than Y=state-of-the-art) Experiment design (measures, data, etc) (e.g., retrieval accuracy on a sample of web data) Test hypothesis (e.g., compare X and Y on the data) Draw conclusions and repeat the cycle of hypothesis formulation and testing if necessary (e.g., X is better only for some queries, now what?) 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

Typical IR Research Process Look for a high-impact topic (basic or applied) New problem: define/frame the problem Identify weakness of existing solutions if any Propose new methods Choose data sets (often a main challenge) Design evaluation measures (can be very difficult) Run many experiments (need to have clear research hypotheses) Analyze results and repeat the steps above if necessary Publish research results 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

Research Methods Exploratory research: Identify and frame a new problem (e.g., “a survey/outlook of personalized search”) Constructive research: Construct a (new) solution to a problem (e.g., “a new method for expert finding”) Empirical research: evaluate and compare existing solutions (e.g., “a comparative evaluation of link analysis methods for web search”) The “E-C-E cycle”: exploratoryconstructiveempiricalexploratory… 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

Types of Research Questions and Results Exploratory (Framework): What’s out there? Descriptive (Principles): What does it look like? How does it work? Evaluative (Empirical results): How well does a method solve a problem? Explanatory (Causes): Why does something happen the way it happens? Predictive (Models): What would happen if xxx ? 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

Look for New IR Research Questions Driven by new data: X is a new type of data emerging (e.g., X= blog vs. news) How is X different from existing types of data? What new issues/problems are raised by X? Are existing methods sufficient for solving old problems on X? If not, what are the new challenges? What new methods are needed? Are old evaluation measures adequate? Driven by new users: Y is a set of new users (e.g., ordinary people vs. librarians) How are the new users different from old ones? What new needs do they have? Can existing methods work well to satisfy their needs? If not, what are the new challenges? What new functions are appropriate for Y? Driven by new tasks (not necessarily new users or new data): Z is a new task (e.g., social networking, online shopping) What information management functions are needed to better support Z? Can these new functions reduced to old ones? If not, what are the new challenges? 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

Solid and High Impact Research Solid work: A clear hypothesis (research question) with conclusive result (either positive or negative) Clearly adds to our knowledge base (what can we learn from this work?) Implications: a solid, focused contribution is often better than a non-conclusive broad explorations High impact = high-importance-of-problem * high-quality-of-solution high impact = open up an important problem high impact = close a problem with the best solution high impact = major milestones in between Implications: question the importance of the problem and don’t just be satisfied with a good solution, make it the best 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

Where to Publish IR Papers Core IR conferences: ACM SIGIR, ACM CIKM ECIR, AIRS Core IR journals ACM TOIS, IRJ IPM, JASIS Web Applications WWW, WSDM Other related conferences Natural Language Processing: HLT, ACL, NAACL, COLING, EMNLP Machine Learning: ICML, NIPS Data Mining: KDD, ICDM Databases: SIGMOD, VLDB, ICDE … 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

Your Assignment Write a team research proposal in response to the “course request for proposals (RFP)” The 5-5 rules: 5 pages (single column, 10 points or 11 points font) Up to 5 team members At least 5 references 5 components (to be described on the next slide) Due 5 days after the lecture period 2-page outline to be finished within the lecture period 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

5 Elements of a Research Proposal Problem statement Motivation: Raise the research question Define the question Benefit of solving the problem Previous/Related work What do we already know about the problem? How would the proposed research increase our knowledge? Research plan How do you plan to solve the problem? Justify your plan (analytical, preliminary results) Evaluation plan Research management Schedule, milestones, fallback plans Resource needs 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008

Next Lecture: Prepare Yourself to do IR research 2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, 2008