1. 1 Beyond Relevance Judgments: Cognitive Shifts and Gratification Professor Amanda Spink, Frances Alvarado-Albertorio & Jia T. Du Queensland University of Technology

2. 2 Introduction Exploring levels of evaluation by IR system users and impacts of IR interaction. Relevance level Cognitive shift level Gratification level

3. 3 Cognitive Shifts Level Cognitive shift or shift in cognitive focus as triggered by the brain’s response and change due to some external force. A shift in cognitive focus (Jacob, 2002) Previous related interactive IR studies – moves (Fidel, 1985), shifts (Robins, 2000; Xie, 2000; Stanton, 2003). Spink (2002) showed that all users experience some level of cognitive shifts – proposed IR evaluation measure information problem shift.

4. 4 Cognitive Shifts Spink and Dee (2007) elucidate the interactive IR as a cognitive shifting process. All study participants reported some level of cognitive shift in their information problem, information seeking, and personal knowledge due to their Web search interaction, positive or negative shift. Different study participants reported different levels of cognitive shift. Some study participants’ reported cognitive shifts in various user-based variables, such as information problem or information seeking stage.

5. 5 Cognitive Shifts Interactive IR involves levels of cognitive shift in information problem, personal knowledge, or information seeking stage. Some study participants reporting that on some levels they remained in the same stage, while shifting on other levels. Phenomenon of cognitive shift is an important element of interactive IR. Conducting further studies examining cognitive shifts, multitasking and cognitive coordination during interactive IR.

6. 6 Gratification Level Gratification is a basic element of human interaction with media and technologies Uses and gratification theory used in media and technology studies (Flanagin & Metzger, 2001). Evidence of users delaying gratification and immediate gratification during interactive processes (Mischel & Ayduk, 2004; Spink, Park & Koshman, 2006). Spink, Park and Koshman (2006) show that Web search ordering by users a delayed gratification or immediate gratification process.

7. 7 Gratification Delayed gratification occurs when users choose to work on harder and low domain knowledge topics first. Immediate gratification occurs when the user chooses to complete easier or high domain knowledge topics first. Need for further research exploring the role of gratification during interactive information retrieval.

8. 8 Further Research Explore the nature and role of various processes during interactive IR that can form the basis of evaluation measures, including: -Cognitive shifts and impacts during interactive IR -Gratification impacts during interactive IR Need to further explore the cognitive impacts of interactive IR beyond relevance judgments.

9. 9 References Du, J. T. (2009). Multitasking, Cognitive Coordination and Cognitive Shifts During Web Searching. Unpublished dissertation, Queensland University of Technology. Spink, A. (2002). A user centered approach to the evaluation of Web search engines: An exploratory study. Information Processing and Management, 38(3), 401-426. Spink, A., & Dee, C. (2007). Cognitive shifts related to interactive information retrieval: A replication study. Online Information Review, 31(6).

10. 10 QUESTIONS? Thank You