2. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 1 THEORETICAL ISSUES IN CONTEXT MODELLING Kalervo Järvelin (kalervo.jarvelin@uta.fi) University of Tampere Finland

3. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 2 Talk Outline  Lab IR - traditional study designs and contributions  Contexts of IR  Modeling context - issues in modeling  A cognitive approach to modeling  Consequent research designs  Conclusion - does it matter?

4. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 3 The Lab IR Framework Docu- ments Docu- ments Represen- tation Represen- tation Database Search request Search request Query Matching Represen- tation Represen- tation Query Result Query Result Evaluation Result Evaluation Result Evaluation Relevance assessment Relevance assessment Recall base Recall base

5. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 4 The most popular focus - it is CS, after all, isn’t it?! IR Research Goals  Traditional goals:  (a) theoretically (~formally) understanding IR,  (b) empirically testing IR methods, and  (c) designing better IR systems  This can be seen in typical IR study designs - by populating the framework with variables

6. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 5 Lab IR Study Types / Designs  Study types:  experimental  methodological  constructive  theoretical  Experimental designs – IR evaluation  focus areas with variables  designs operate with these  fringe areas controlled (no analytical variables)  dependent variables: E R  many hidden variables

7. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 6 LabIR: Hypos, Laws, Theories  The dependent variables - typically recall and precision - using various metrics  The independent variables typically are the use or non-use of various techniques  The controlled variables - collection, requests  Thus Lab IR is about the explanation of the variation of recall and precision through IR techniques applied

8. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 7 LabIR: Contributions  IR technology  Theoretical growth  theory expansion, e.g., due to enrichment of indexing theory  greater analytical power through formal model building  improved empirical support through testing over several test collections  new research questions – within the given set of variables  Theoretical growth has been limited by not bringing in radically new concepts or relations  Cannot possibly analyze real world applicability

9. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 8 Lab IR Tests  IR research typically considers only retrieval tasks which most often are:  (a) purely topical  (b) content-only  (c) well-defined  (d) static, and  (e) exhaustive This is like saying that no matter what your situation is, your needs always are purely topical, content-only, well-defined, static, and … … True, isn’t it?

10. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 9 Small World?

11. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 10 Psssst...... IR in Context Alternatives?

12. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 11 One possible context: Task Performance Augmentation through IR Augmenting Task Performance Colleagues KnowledgeMethodology Education/Training Create Information Seeking Remember/Use Tools Work Task Situation Other Sources Education Documents Information Systems Acquire Knowledge Find Docs Docs Available Retrieve Docs Other Collections Colleagues... DocDBFactDB Context Web...

13. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 12 Socio-organizational& cultural context Nested Context Frameworks  How much should one cover? Work Task Work Process Task Result Work task context Seeking Task Seeking Process Seeking Result Seeking context Docs Repr DB Request Query Match Repr Result IR context  How to model what is covered?

14. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 13 What to cover?  Clearly, current IR test designs are not representative of real world IR  In principle,  whatever has important relationships with the application of IR, should be covered  however, what has, is not well known  answering this requires theoretical modeling, operationalization and empirical testing  so, what is context?

15. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 14 Context? What?!  Dervin (1997):  there is no term that is more often used, less often defined, and when defined defined so variously as context - it has become almost a ritualistic invocation  for some, context has the potential of being virtually anything that is not defined as the focus  for others, it is inextricable surround that denies all generalizations  there are endless lists of contextual factors  This is why we hate it - it is foreign to CS!

16. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 15 Some Contextual Themes  Reality is discontinuous, knowledge partial  Context is not useful as an independent entity  Context requires a focus on process and relationships between product and process  Need to focus on multiple interdependencies  Context is a necessary source of meaning (Dervin 1997)

17. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 16 How to model? - IR Context Dimensions 1.Work task dimension 2.Search task dimension 3.Actor dimension 4.Perceived work task dimension 5.Perceived search task 6.Document dimension 7.Algorithmic search engine dimension 8.Algorithmic interface dimension 9.Access and interaction dimension Each dimension containing multiple variables Much more detail in the forthcoming book by Peter&Kal

18. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 17 http://www.springeronline.com/

19. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 18 Information Objects Information Systems Interfaces Cognitive Actor(s) Organisat. Cultural = Cognitive transformation and influence = Interactive communication of cognitive structures Social & 4 3 2 1 5 6 8 7 Context A Cognitive Model of IS&R

20. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 19 Context around and within Information objects IT: Engines Algorithms Interface Information seeker Org. Cultural Social Context Real Social & Physical World Cognitive World Perceived Information objects Perceived IT: Engines Algorithms Perceived Interface Situation i.e. perceived task – need sources Social Org. / Cultural Perceived context

21. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 20 Augmented IR Research Goals  Theoretically understanding the application of IR in / to context ; not just engines but also their use in varying situations  Empirically describing / testing the application of IR in / to various contexts  Supporting the design of information systems, services and information management  This does not downgrade IR into Social Science but rather means more / better engineering

22. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 21 New Research Settings  Extended laboratory experiments  What kinds of work tasks are typical in Domain X?  What kinds of search tasks do these generate?  What kinds of documents tend to be relevant?  IIR experiments  Lab approach but including the seeking actor  Research on interfaces / actors in context  interaction of actors and information system interfaces in diverse socio-organizational contexts

23. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 22 Lab IR settings focused on domain/task variables and dimensions of IT and information objects Information objects IT Task Org. Context Interface Actor(s) Social Cultural

24. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 23 Lab IR settings focusing on actor variables and dimensions of IT and information objects Information objects IT Task Org. Context Interface Social Cultural Work task perception Search task perception Actor types

25. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 24 Conclusion  Context is endless and pervasive  its management requires perspective, goals and interests: what do we want to achieve??  we may see multiple, nested contexts  Thus, what is IR about? As technology...  developing systems based on a technological imperative? based on people’s communication / access needs?  As a discipline... [vs. a technology]  understanding... perhaps... but what? C-n-c-l?

26. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 25 Does context really matter?  We do not know really (if, or how much)  Some certainly wish / believe it does not  I don’t believe them until shown that  no matter what one’s situation is, one’s information needs are of just one type  no matter of what type information needs are, they are always best served in one already known way

27. Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 26