Design of metadata surrogates in search result interfaces of learning object repositories: Linear versus clustered metadata design Panos Balatsoukas Anne Morris Ann O’Brien

Contents  Definition of user-centred metadata  Evolution of metadata surrogate design  Aim and Objectives of the usability test  The META-LOR 1 prototype  Methodology  Results  Conclusions – Recommendations  Future Research

Metadata definitions  ‘Data about data’  “Structured data about an object that supports functions associated with the designated object” – (Greenberg, 2005)  Learning object metadata: metadata used for the efficient description of learning objects and the effective support of educational-learning functions related to the described learning objects.

User-centred metadata ContentPresentation Learner Relevance Usability Technologies

Evolution of metadata presentation and content

Copied from: Marchionini et al (1993)

Design of metadata surrogates  Metadata elements providing access or arranging access to the resource should follow content related elements such as title, abstract, subject heading or keywords.  Users prefer content related metadata for finding and identifying sources and technical and physical metadata for selecting and obtaining access to the resource.  There is a debate among researchers as to whether metadata surrogates should be displayed in list, tabular, dynamic or category-based format in search result interfaces.  It is suggested that abstracts should contain contextualised information relevant to users’ search query.  Metadata surrogates should not include only topical or subject related information.

Design of learning object metadata in search result interfaces  The need to include a description/abstract of the contents of the learning object in the metadata surrogate;  The use of user-centred metadata terminology and vocabularies; and  The use of clustered rather than linear and information cluttered learning object metadata surrogates.

Aims and objectives  To examine users’ interaction with two different learning object metadata surrogates: 1. a linear metadata surrogate interface, and 2. a clustered metadata surrogate interface.  The objectives of this study were:  To investigate the time needed by learners to identify a relevant learning object, using both interfaces;  To study the impact of task complexity on users’ interaction with both interfaces; and  To examine learners’ subjective satisfaction for both interfaces.

Linear metadata surrogate Pop up box

Clustered metadata surrogate Metadata categories

Methodology 1  Usability participants’ profile:  12 postgraduate students in Information and Computer Science  Task List analysis and scenarios:  3 tasks with varying degrees of complexity (Low, Medium and High complexity)  Error rate and Time  Observation (Think Aloud protocol)  Background and post test questionnaires  Post test interviews

Methodology 2 The three Tasks:

Results of the usability test

Differences in Time  participants performed the three tasks slightly faster using the clustered metadata surrogate interface.  Mean time of 314 secs in the Linear.  Mean time of 301 in the Clustered

Task complexity and Interface  There were no significant differences observed between task complexity and metadata interface design

Subjective satisfaction  Subjects were significantly more satisfied with the clustered metadata surrogate interface.  Mean overall satisfaction for clustered metadata surrogate = 7.8.  Mean overall satisfaction for the linear metadata surrogate interface = 6.3

Qualitative results (1)  Participants (n=10) liked the way metadata was presented in the clustered metadata surrogate interface:  Plausibility and engagement  Structure and organisation of information  Two participants preferred the linear interface (prior familiarisation; not meaningful metadata clustering).

Qualitative results (2)  Subjects liked the use of most of the general and technical category metadata (e.g. title, subject, description, format, identifier)  Few of the educational related metadata were perceived as useful (e.g. Audience, interactivity level, difficulty)

Qualitative results (3)  Subjects did not like the inclusion of many metadata elements and lengthy metadata surrogates.  Some participants (n=4) would like to select the metadata elements to be displayed in the surrogate.  Other metadata elements:  Relation metadata  People’s comments  The time it takes for a learning object to be downloaded/accessed  Accessibility needs  Information about the quality of learning objects

Conclusions - Recommendations  The provision for alternative displays of metadata surrogates, for example, both in linear and clustered forms.  The design of adaptive interfaces that present the content and format of metadata surrogates according to learners’ needs.  The use of pop up boxes for documenting and presenting the meaning of learning object metadata elements to users.  Need to extend the LOM standard with new metadata elements, such as, ‘the time it takes for a learning object to be downloaded’, ‘accessibility needs’ information, as well as ‘information about the ‘quality’ of a learning object.

Research in progress…  Usability assessment of three learning object repositories (MERLOT, ARIADNE Knowledge Pool and JORUM/UK).  Survey of students’ perceptions of the importance of learning object metadata elements.  User study on the criteria students employ to judge the relevance of learning objects.  Development of Heuristic evaluation checklist for the evaluation of metadata surrogates in search and search result interfaces.  Development of guidelines and recommendations for the design of learning object metadata schemas and Learning Object Repositories.