1. Implicit Acquisition of Context for Personalization of Information Retrieval Systems Chang Liu, Nicholas J. Belkin School of Communication and Information Rutgers University imliu@gmail.com, belkin@rutgers.edu imliu@gmail.com

2. Research Goal Propose a model to personalize search results according to the user’s search context, in particular the type of task that led the user to engage in information-seeking behavior, and the behaviors that the user has engaged in during the search. This personalization is to predict potentially useful documents based on the type of task, and on behaviors indicative of document usefulness. 2

3. Sources of Evidence of Usefulness for Implicit Relevance Feedback Measures of behaviors on content pages –Time-related measures: display (dwell) time –Amount of actions: number of scrolling, mouse movement & clicks, number of visits –Further usage of content pages: print, bookmark, save, etc. –Patterns of eye movements Measures of behaviors on search result pages –Click-through, click order, click position, number of clicks –Time on result list before first click, total time on result lists; –Query Reformulation Interval time; –Query Reformulation Type Query logs –Previous query issuing and results browsing behaviors Combination of multiple behavioral measures 3

4. Task Type as Context Task type can influence –interpretation of user behaviors for implicit relevance feedback (White and Kelly, 2006; Liu and Belkin, 2010) –influence users’ search behaviors (Li, 2008; Toms et al. 2008; Kim, 2009) –influence the type of information objects users expect (Freund, 2008) –the performance of personalization algorithms may vary for different types of information goals (Teevan, Dumais and Horvitz, 2010) –Classification: Task product, Complexity, Goal (quality), Level of document judgment, etc. Detect task type from user behaviors 4

5. PooDLE Project (Design) Task-Cognitive Experiment (TCE) –A lab-based user study (journalism domain) –Four search tasks were designed based on task facets –32 participants Domain-Knowledge Experiment (DKE) –A lab-based user study (medical domain) –Five search tasks from 2004 TREC Genomics track –Questionnaire about participants’ background information, domain knowledge, search knowledge, etc. –40 participants Both experiments –A variety of searcher behaviors: eye gaze, various interactions with the search systems and information objects, saving and deleting pages, evaluation of usefulness of the saved pages, etc. 5

6. PooDLE Project (Some Results) Task type and search behaviors –Task type affected average decision time and the ratio of reading to scanning in participants’ reading models –Some within-session behaviors could indicate the difficulty of search tasks Search behaviors and document usefulness –Query reformulation intervals were related to whether users found useful document(s) in the intervals or not –Query reformulation type –Knowledge of task stage can help in inferring document usefulness from decision time, especially in the parallel task. 6

7. Investigation Plan Three-part plan Start –Identify salient behaviors –Generate predictive models –Test Second stage –Implement personalized IR based on first stage –Test Third stage –Evaluate personalization in experimental setting 7

8. Activate IR algorithm Generate Predictive Models Predictive Model of the Usefulness of Content Pages Analyze measures of user behaviors in user studies A general predictive model (The default model) Specific predictive models for each type of tasks Re-rank search results /Reformulate queries Implement personalized IR algorithm in the system Evaluation of the IR system Observe user behaviors Can it predict the task type? Not yet Yes Comparative study Personalized IR system Non- Personalized IR system V.S. Predictive Model of Task Type

9. Conclusion We propose a program for developing and evaluating a personalized IR model –collect user behaviors –predict task type and documents useful to the tasks –personalize search results –evaluate to see if this leads to a better search experience 9

10. Acknowledgments The research that led to this proposal was supported by IMLS Grant LG-06-07-0105-07. We owe a great debt to our colleagues in the PoODLE project*, Michael Cole, Jingjing Liu, Ralf Bierig, Jacek Gwizdka, Jun Zhang and Xiangmin Zhang. *http://comminfo.rutgers.edu/imls/poodle 10