IR2001 Oulu Vakkari & Serola & Pennanen1 The impact of the anticipated contribution of information on search tactics and results in task performance Pertti Vakkari & Sami Serola & Mikko Pennanen Department of Information Studies University of Tampere Finland

IR2001 Oulu Vakkari & Serola & Pennanen2 Background People use information systems in order to find contributing information for their task at hand Their expectations of what kind of information might contribute steer –The choice of search terms and tactics in query formulation –The evaluation of search results Complex task require successive searches over time The anticipated contribution of information varies according to the stage of the task process –The choice of terms and tactics vary according to the process

IR2001 Oulu Vakkari & Serola & Pennanen3 Earlier studies 1 No direct empirical, mostly analytical evidence Users’ position in the problem treatment process has an impact on searching (Belkin & Seeger & Wersig 1983) Models incorporating users’ current work task and searching (Sutcliffe & Ennis 1998; Vakkari 2001) –Leave open how anticipated contribution of information searched for is related to search tactics and terms

IR2001 Oulu Vakkari & Serola & Pennanen4 Earlier studies 2 Kuhlthau’s model predicts that people need, search for and use information differently depending on the stage of the task –In the beginning before focus formulation general background information is looked for, after focus formulation pertinent, specific information is searched for The stage of the research proposal preparation by the students differentiated the type of information found as contributory, and search tactics (Vakkari 2000a,b) –As the process proceeded, the use of background information and theories declined whereas the use of methods, empirical results and specific information grew –As the process proceeded, the number and specificity of search terms increased; the number and type of search tactics used increased

IR2001 Oulu Vakkari & Serola & Pennanen5 Data 22 undergraduates of psychology preparing a research proposal for a small empirical study during a seminar of three months –18 attended a short training session for PsychoINFO (a major bibliographical info system in ps.) (no differences between the groups) A search in PsychoINFO in the beginning and end of the seminar (pre-focus- post-focus stages/Kuhlthau) –Think aloud during search sessions and search logs recorded A presession interview –Kuhlthau’s process survey questionnaire & (semi-)structured interv. Feelings, thoughts and actions State of knowledge and experience of the topic Search goals and the expected use of results A post session interview –Assessment of accepted references

IR2001 Oulu Vakkari & Serola & Pennanen6 Concepts Stages in Kuhlthau’s (1993) model used to indicate the students’ stages in task performanceStages in Kuhlthau’s (1993) model used to indicate the students’ stages in task performance A query consists of terms and operatorsA query consists of terms and operators A facet is an exclusive aspect of a queryA facet is an exclusive aspect of a query A move is an identifiable thought or action for improving search resultsA move is an identifiable thought or action for improving search results –A move is a change made in a query in order to attain the goals of the search (in this study) Tactics consist of a set of moves

IR2001 Oulu Vakkari & Serola & Pennanen7 Tactics

IR2001 Oulu Vakkari & Serola & Pennanen8 Types of contributing information General information –General background information of the topic (G) –Theories, models, the definitions of concepts (T) Specific information –Information about research methods (M) –Particular empirical research results (E) –Particular specific information (P) D = (B+T/2) - (M+E+P/3) –Two groups formed in the first and second round based on D –I round: expected contribution: general or mixed –II round: expected contribution: mixed or specific

IR2001 Oulu Vakkari & Serola & Pennanen9 The average number of terms and facets per student in the 1st session

IR2001 Oulu Vakkari & Serola & Pennanen10 The average number of tactics per student in the 1st session NT = a narrower term; RT = a related term Students needing mixed information worked harder in their searches Their wider search vocabulary partly explains their greater number of tactics (r =.73) Vary tactics = replacing of terms a clumsy tactic for compensating parallel tactic (the use of OR operator) for increasing recall

IR2001 Oulu Vakkari & Serola & Pennanen11 The average number of useful references per student in the 1st session Those needing general information made less effort in searching, but found more useful references Although both groups were at the same stage in Kuhlthau’s model, there might be differences in the familiarity with the topic between the groups Those more knowledgeable tend to accept fewer references as relevant (Spink & al. 1998, Vakkari & Hakala 2000)

IR2001 Oulu Vakkari & Serola & Pennanen12 The average number of terms and facets per student in the 2nd session

IR2001 Oulu Vakkari & Serola & Pennanen13 The average number of tactics per student in the 2nd session The students looking for mixed information were using more tactics, especially vary tactics In the course of the seminar the students did not utilized OR operator for expressing terms in facets

IR2001 Oulu Vakkari & Serola & Pennanen14 The average number of useful references per student in the 2nd session The greater effort put in searching by the students needing mixed information led to a better search result They were not as advanced in the preparation of the proposal in terms of Kuhlthau’s model as their fellow students (p=.02) Those less knowledgeable tend to accept more references as relevant (Spink & al. 1998, Vakkari & Hakala 2000) Is the better search result a reflection of the use of more search terms and tactics or of the tendency of less knowledgeable to accept easier relevant references ?

IR2001 Oulu Vakkari & Serola & Pennanen15 Conclusions The type of the information needed depended on the stage of the process –The results consistent with Vakkari (2000) The number of useful references found seems to depend more on the novice searchers’ ability to assess references (domain knowledge) than on the number of search terms and tactics used, if they are novices in a domain Although vary tactic is a clumsy way of substituting parallel tactics, it is used for exploring the conceptual space of the topic