2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, Introduction to IR Research ChengXiang Zhai Department of Computer Science Graduate School of Library & Information Science Institute for Genomic Biology, Statistics University of Illinois, Urbana-Champaign

2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, What is Research? Research –Discover new knowledge –Seek answers to questions Basic research –Goal: Expand man’s knowledge (e.g., which genes control social behavior of honey bees? ) –Often driven by curiosity (but not always) –High impact examples: relativity theory, DNA, … Applied research –Goal: Improve human condition (i.e., improve the wolrd) (e.g., how to cure cancers?) –Driven by practical needs –High impact examples: computers, transistors, vaccinations, … The boundary is vague; distinction isn’t important

2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, Why Research? Amount of knowledge Advancement of Technology Utility of Applications Quality of Life Basic Research Applied Research Application Development Curiosity

2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, Where’s IR Research? Amount of knowledge Advancement of Technology Utility of Applications Quality of Life Basic Research Applied Research Application Development Information Science Computer Science

2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, Where’s Your Position? Amount of knowledge Advancement of Technology Utility of Applications Quality of Life Basic Research Applied Research Application Development Different position benefits from different collaborators

2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, Research Process Identification of the topic (e.g., Web search) Hypothesis formulation (e.g., algorithm X is better than Y=state-of-the-art) Experiment design (measures, data, etc) (e.g., retrieval accuracy on a sample of web data) Test hypothesis (e.g., compare X and Y on the data) Draw conclusions and repeat the cycle of hypothesis formulation and testing if necessary (e.g., Y is better only for some queries, now what?)

2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, Typical IR Research Process Look for a high-impact topic (basic or applied) New problem: define/frame the problem Identify weakness of existing solutions if any Propose new methods Choose data sets (often a main challenge) Design evaluation measures (can be very difficult) Run many experiments (need to have clear research hypotheses) Analyze results and repeat the steps above if necessary Publish research results

2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, Research Methods Exploratory research: Identify and frame a new problem (e.g., “a survey/outlook of personalized search”) Constructive research: Construct a (new) solution to a problem (e.g., “a new method for expert finding”) Empirical research: evaluate and compare existing solutions (e.g., “a comparative evaluation of link analysis methods for web search”) The “E-C-E cycle”: exploratory  constructive  empirical  exploratory…

2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, Types of Research Questions and Results Exploratory (Framework): What’s out there? Descriptive (Principles): What does it look like? How does it work? Evaluative (Empirical results): How well does a method solve a problem? Explanatory (Causes): Why does something happen the way it happens? Predictive (Models): What would happen if xxx ?

2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, Look for New IR Research Questions Driven by new data: X is a new type of data emerging (e.g., X= blog vs. news) –How is X different from existing types of data? –What new issues/problems are raised by X? –Are existing methods sufficient for solving old problems on X? If not, what are the new challenges? –What new methods are needed? –Are old evaluation measures adequate? Driven by new users: Y is a set of new users (e.g., ordinary people vs. librarians) –How are the new users different from old ones? What new needs do they have? –Can existing methods work well to satisfy their needs? If not, what are the new challenges? –What new functions are appropriate for Y? Driven by new tasks (not necessarily new users or new data): Z is a new task (e.g., social networking, online shopping) –What information management functions are needed to better support Z? –Can these new functions reduced to old ones? If not, what are the new challenges?

2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, Solid and High Impact Research Solid work: –A clear hypothesis (research question) with conclusive result (either positive or negative) –Clearly adds to our knowledge base (what can we learn from this work?) –Implications: a solid, focused contribution is often better than a non-conclusive broad explorations High impact = high-importance-of-problem * high-quality-of- solution –high impact = open up an important problem –high impact = close a problem with the best solution –high impact = major milestones in between –Implications: question the importance of the problem and don’t just be satisfied with a good solution, make it the best

2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, Where to Publish IR Papers Core IR conferences: –ACM SIGIR, ACM CIKM –ECIR, AIRS Core IR journals –ACM TOIS, IRJ –IPM, JASIS Web Applications –WWW, WSDM Other related conferences –Natural Language Processing: HLT, ACL, NAACL, COLING, EMNLP –Machine Learning: ICML, NIPS –Data Mining: KDD, ICDM –Databases: SIGMOD, VLDB, ICDE …

2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, Your Assignment Write a team research proposal in response to the “course request for proposals (RFP)” The 5-5 rules: –5 pages (single column, 10 points or 11 points font) –Up to 5 team members –At least 5 references –5 components (to be described on the next slide) –Due 5 days after the lecture period 2-page outline to be finished within the lecture period

2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, Elements of a Research Proposal Problem statement –Motivation: Raise the research question –Define the question –Benefit of solving the problem Previous/Related work –What do we already know about the problem? –How would the proposed research increase our knowledge? Research plan –How do you plan to solve the problem? –Justify your plan (analytical, preliminary results) –Evaluation plan Research management –Schedule, milestones, fallback plans Resource needs

2008 © ChengXiang Zhai Dragon Star Lecture at Beijing University, June 21-30, Next Lecture: Prepare Yourself to do IR research