End-user interaction with corporate digital thesaurus Marianne Lykke Nielsen The Royal School of Library and Information Science Department of Information.

Slides:

Advertisements

Similar presentations

©2011 1www.id-book.com Evaluation studies: From controlled to natural settings Chapter 14.

Advertisements

Chapter 14: Usability testing and field studies

The Feasibility of Using the Semantic Components Model for Indexing Documents in Digital Libraries * Susan Price + Marianne Lykke Nielsen * Lois Delcambre.

Indexing challenges in work place information retrieval Marianne Lykke Nielsen & Anna Gjerluf Eslau NKOS 2006 Controlled, human indexing vs full-text indexing.

“Do users do what they think they do?” – a comparative study of user perceived and actual information searching behaviour in the National electronic Library.

Developing and Evaluating a Query Recommendation Feature to Assist Users with Online Information Seeking & Retrieval With graduate students: Karl Gyllstrom,

HOW DO PROFESSIONAL DEVELOPERS COMPREHEND TO SOFTWARE Report submitted by Tobias Roehm, Rebecca Tiarks, Rainer Koschke, Walid Maalej.

Leveraging Your Taxonomy to Increase User Productivity MAIQuery and TM Navtree.

Chapter 14: Usability testing and field studies. 2 FJK User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.

Text Operations: Preprocessing. Introduction Document preprocessing –to improve the precision of documents retrieved –lexical analysis, stopwords elimination,

© Tefko Saracevic, Rutgers University1 digital libraries and human information behavior Tefko Saracevic, Ph.D. School of Communication, Information and.

© Tefko Saracevic, Rutgers University1 Search strategy & tactics Governed by effectiveness & feedback.

Chapter 14: Usability testing and field studies. Usability Testing Emphasizes the property of being usable Key Components –User Pre-Test –User Test –User.

© Tefko Saracevic, Rutgers University 1 EVALUATION in searching IR systems Digital libraries Reference sources Web sources.

© Tefko Saracevic, Rutgers University1 digital libraries and human information behavior Tefko Saracevic, Ph.D. School of Communication, Information and.

IR2001 Oulu Vakkari & Serola & Pennanen1 The impact of the anticipated contribution of information on search tactics and results in task performance.

Educational Research Methods Aims: You should be able to - devise a suitable research question; - locate research questions and methods in terms of research.

Experimental Components for the Evaluation of Interactive Information Retrieval Systems Pia Borlund Dawn Filan 3/30/04 610:551.

Enterprise social bookmarking - in a community of practice in IBM, Denmark by Joachim Florentz Boye and Marianne Lykke Nielsen Royal School of Library.

Copyright c 2001 The McGraw-Hill Companies, Inc.1 Chapter 2 The Research Process: Getting Started Researcher as a detective Seeking answers to questions.

From Controlled to Natural Settings

© Tefko Saracevic, Rutgers University1 digital libraries and human information behavior Tefko Saracevic, Ph.D. School of Communication, Information and.

Research problem, Purpose, question

Creating Knowledge V, 2008 A search thesaurus for the domain of linguistics Creating a domain specific search tool on the basis of user behaviour study.

Qualitative Research Interviews Josh Fiala DIS /5/08.

The Role of Automated Categorization in E-Government Information Retrieval Tanja Svarre & Marianne Lykke, Aalborg University, DK ISKO conference, 8th of.

An Introduction to Research Methodology

Preliminary Survey Lectured by Dr. Siriluck Sutthachai Accounting Department Faculty of Management Science Khon Kaen University Khon Kaen, Thailand.

Languages are bridges … not barriers Chiara Carlucci – CEDEFOP Library ReferNet Technical Meeting September 2009.

August 21, 2002Szechenyi National Library Support for Multilingual Information Access Douglas W. Oard College of Information Studies and Institute for.

Information Seeking in Electronic Environments Marchionini, G. (1995). Information Seeking in Electronic Environments. Cambridge Press. Kathleen Padova.

Qualitative Analysis Information Studies Division Research Workshop Elisabeth Logan.

Evaluation Experiments and Experience from the Perspective of Interactive Information Retrieval Ross Wilkinson Mingfang Wu ICT Centre CSIRO, Australia.

How doctors apply semantic components to specify search in work-related information retrieval Marianne Lykke, professor, Aalborg University Susan Price,

H. Lundbeck A/S3-Oct-151 Assessing the effectiveness of your current search and retrieval function Anna G. Eslau, Information Specialist, H. Lundbeck A/S.

User Interface Design Chapter 4 User requirements: elicitation and analysis.

3. Qualitative Research. Exploratory Research When a researcher has a limited amount of experience with or knowledge about research issue, exploratory.

Usability testing. Goals & questions focus on how well users perform tasks with the product. – typical users – doing typical tasks. Comparison of products.

Selection and Definition of a Problem. First Step Identify a general area that is related to your area of expertise and is of particular interest to you.

Planning a search strategy.  A search strategy may be broadly defined as a conscious approach to decision making to solve a problem or achieve an objective.

Marianne Lykke Nielsen September 2008 Indexing with semantic components improve information retrieval in domain-specific web portal Marianne Lykke Nielsen.

Qualitative Research Interviews March 25, What Are Qualitative Interviews? “…attempts to understand the world from the subjects' point of view,

Implicit Acquisition of Context for Personalization of Information Retrieval Systems Chang Liu, Nicholas J. Belkin School of Communication and Information.

FOR 500 PRINCIPLES OF RESEARCH: PROPOSAL WRITING PROCESS

Testing & modeling users. The aims Describe how to do user testing. Discuss the differences between user testing, usability testing and research experiments.

Math Information Retrieval Zhao Jin. Zhao Jin. Math Information Retrieval Examples: –Looking for formulas –Collect teaching resources –Keeping updated.

The UNESCO Thesaurus Meeting for Managers of UNESCO Documentation Networks Meron Ewketu UNESCO Library June

1 CS 502: Computing Methods for Digital Libraries Lecture 19 Interoperability Z39.50.

Thesauri usage in information retrieval systems: example of LISTA and ERIC database thesaurus Kristina Feldvari Departmant of Information Sciences, Faculty.

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 1 Chapter 2 The Research Process: Getting Started Researcher as a detective –Seeking answers.

Centre for Information & Knowledge Management INFORMATION SYSTEMS MANAGEMENT Jamie O’Brien Centre for Information & Knowledge Management University of.

Modern Information Retrieval Presented by Miss Prattana Chanpolto Faculty of Information Technology.

Introduction to Information Retrieval Example of information need in the context of the world wide web: “Find all documents containing information on computer.

Jane Reid, AMSc IRIC, QMUL, 30/10/01 1 Information seeking Information-seeking models Search strategies Search tactics.

Marketing Research Process and Types of Marketing Research.

June 2003INIS Training Seminar1 INIS Training Seminar 2-6 June 2003 Subject Analysis Thesaurus and Indexing Alexander Nevyjel Subject Control Unit INIS.

L&I SCI 110: Information science and information theory Instructor: Xiangming(Simon) Mu Sept. 9, 2004.

Digitization of Publications, Technical and Consultancy Reports of Marine Species in Kenya (DRMS) Dr. Nina Wambiji 21 st September, 2012.

Evaluation of an Information System in an Information Seeking Process Lena Blomgren, Helena Vallo and Katriina Byström The Swedish School of Library and.

STEP - 4 Research Design 1. The term “research design” can be defined as, The systematic study plan used to turn a research question or research questions.

Market Research Unit 3 P3.

EnTag Enhanced Tagging for Discovery Koraljka Golub, Jim Moon,

Marketing Information, Research, and Understanding the Target Market

From Controlled to Natural Settings

The MR process.

Evaluation of IR Performance

digital libraries and human information behavior

Testing & modeling users

BBA V SEMESTER (BBA 502) DR. TABASSUM ALI

Presentation transcript:

End-user interaction with corporate digital thesaurus Marianne Lykke Nielsen The Royal School of Library and Information Science Department of Information Studies NKOS 21. August 2003

NKOS, August 2003Marianne Lykke Nielsen Topic The aim is to present findings concerning user behaviour and user interaction with a corporate, digital thesaurus and discuss implication for thesaurus design. Two user studies were carried out in a large Danish pharmaceutical company investigating searching behaviour and information seeking behaviour. The first study formed part of preliminary system analysis determining scope and content of the corporate thesaurus. The second study made up the later evaluation of the thesaurus.

NKOS, August 2003Marianne Lykke Nielsen Outline Design of user studies Findings Implications for thesaurus design

NKOS, August 2003Marianne Lykke Nielsen Design of user study I Objective - Gain insight into the context of the information environment: - Work tasks, problem situations, information needs - Position, educational background, work and search experience - Information resources - Searching behaviour and searching problems Data collection methods - Group interviews, 29 respondents - Content and discourse analysis of 50 user requests

NKOS, August 2003Marianne Lykke Nielsen Work tasks/ Discourses A priori determinability RepetitivenessNovelty of topicType and structure Basic researchLow degree of determinability Low repetitiveness Known and unknown concepts Conscious information need 1 – 2 interrelated facets Clinical and non- clinical testing Low to medium degree of determinability Some repetitiveness Known conceptsConscious information need 3 or more interrelated facets Sale and CIMedium degree of determinability High repetitiveness Known conceptsConscious information need 3 or more interrelated facets Information needs

NKOS, August 2003Marianne Lykke Nielsen

NKOS, August 2003Marianne Lykke Nielsen

NKOS, August 2003Marianne Lykke Nielsen Design of user study II (1/3) Objective - Investigation of user interaction and user satisfaction with corporate thesaurus: - Problem analysis - Query (re)formulation - Understanding of search concepts - Lead-in - Experimental retrieval test - 20 researchers from R&D - Controlled search jobs - In-house retrieval system, - commando-based, exact match system (BRS/Search) - semi-controlled, manual indexing practice

NKOS, August 2003Marianne Lykke Nielsen Design of user study II (2/3) Test procedure – before search sessions: 1. Investigation of searcher skills by questionnaire Test procedure - during two-hour test session: 1.Introductory orientation session 2.Pre-search investigation of searcher´s mental model of search problem and expectations by questionnaire and interview 3.Search session and investigation of user actions by observation and log-file 4.Post-search investigation of user motivation and satisfaction by interview and questionnaire Step 2 – 4 were replicated for each search problems.

NKOS, August 2003Marianne Lykke Nielsen Data types and collection methods (3/3) Qualitative data Mental model of search task by Interview and questionnaire Explanation of search actions by Interview User satisfaction by Interview Quantitative data User skills by Questionnaire User satisfaction by Questionnaire User actions by Logging & observation User satisfaction by Relevance judgements

NKOS, August 2003Marianne Lykke Nielsen Thesaurus use (n = 58) Role in searching Amount of sessions % Problem analysis-- Query formulation5086 Understanding of search concepts 1322 Lead-in3967

NKOS, August 2003Marianne Lykke Nielsen Search moves (1/2) Search formulation movesAmount% Exhaust Specify5021 Parallel4720 Reduce Exclude52 Pinpoint21 Total235100

NKOS, August 2003Marianne Lykke Nielsen Search moves (2/2) Search term movesAmount% Vary10973 Fix2618 Respell85 Repeat64 Respace-- Total149100

NKOS, August 2003Marianne Lykke Nielsen Sources for search terms SourceAmount% Search problem19046,6 Thesaurus17843,6 User interaction389,3 Records20,5 Total408100

NKOS, August 2003Marianne Lykke Nielsen Type of search terms RelationshipsAmount% Synonym (SYN)8145,5 Preferred term (PT)5229,2 Narrower term (NT)3519,7 Related term (RT)73,9 Broader term (BT)31,7 Total178100

NKOS, August 2003Marianne Lykke Nielsen Work task dependent behaviour (1/4) Basic research scientists (n = 32) Clinical research scientists (n = 26) TotalAverageTotalAverage Search runs2086,51254,8 Thesaurus look-ups1153,61013,9

NKOS, August 2003Marianne Lykke Nielsen Work task dependent behaviour (2/4) Basic research scientists (n = 32) Clinical research scientists (n = 26) Search movesTotalAverageTotalAverage Parallel361,1110,4 Vary621,9471,8

NKOS, August 2003Marianne Lykke Nielsen Work task dependent behaviour (3/4) Basic research scientists (n = 32) Clinical research scientists (n = 26) Sources for search terms Total% % Search problem Thesaurus User interaction Record21-- Total

NKOS, August 2003Marianne Lykke Nielsen Work task dependent behaviour (4/4) Basic research scientists (n = 32) Clinical research scientists (n = 26) Type of search termsTotal% % Synonym (SYN) Preferred term (PT) Narrower term (NT) Related term (RT)4337 Broader term (BT)32-- Total

NKOS, August 2003Marianne Lykke Nielsen Summing up implications for design - Strong domain-orientation - Indication of discourses - Redundancy - Explanation - User-controlled interaction - Ease of use

NKOS, August 2003Marianne Lykke Nielsen Future research - Importance and awareness of discourses - Can we ”avoid” controlled indexing in in-house databases and relay on strong thesaurus informing about discourses? - Ease of use - Will structured query modification improve retrieval? - Automatic categorization -Will domain-specific thesaurus improve semiautomatic categorization?

NKOS, August 2003Marianne Lykke Nielsen Literature Nielsen, M L (2002). The word association method. A gateway to work-task based retrieval. Åbo: Åbo Academy University Press. 342 p. Available at: