1. Metadata Quality The GLOBE experience Frans Van Assche Secretary General http://www.globe-info.org/

2. Quality Fitness of purpose as seen by a community Which communities for metadata?  Users of learning resources  Brokers who want to share metadata  Tool providers using metadata

3. Quality Metrics for Users of learning resources Metadata have higher quality to the extend that they improve the effectiveness of  Discovery of a LO  Evaluation & choice of a LO  Procurement of a LO  Obtaining an LO  Modifications of a LO  Technical integration of a LO  Integration of a LO in the classroom activity  Use of a LO

4. Quality Metrics for Brokers of & Tool providers for learning resources Metadata have higher quality to the extend that they improve the effectiveness of  Maintaining metadata (readability, modifyability, …)  Technically exchange metadata – technical interoperability  Interpret tokens in a compatible way or be able to derive tokens in a compatible way – semantic interoperability  Recognising updates of metadata  Supporting users with tools

5. Validation About 100 targets from more than 200 collections About 20 different validation schemes About 2,000,000 LOMs

6. The ARIADNE ingestion process Harvesting Transformation Validation Preprocessing IndexesIMR Registering

9. Validation Service Paste metadata record  e.g LOM instance Select validation scheme uri  e.g http://ltsc.ieee.org/xsd/LOM/stricthttp://ltsc.ieee.org/xsd/LOM/strict Validate & Consult Validation Errors Based on XSD Schema & Schematron Rules http://bit.ly/90va7d 9 In markup languages, Schematron is a rule-based validation language for making assertions about the presence or absence of patterns in XML trees. It is a structural schema language expressed in XML using a small number of elements and XPath.markup languagesage for maral In markup languages, Schematron is a rule-based validation language for making assertions about the presence or absence of patterns in XML trees. It is a structural schema language expressed in XML using a small number of elements and XPath.markup languagesage for maral

10. Example Validation Service http://bit.ly/90va7d

11. Learning Resource Type

12. Analysing the GLOBE metadata set Change over time of the values for general.language

13. Analysing the GLOBE metadata set (2)

14. Analysing the GLOBE metadata set (3)

15. Entropy (cf Shannon)

16. Percentage of usage Title Identifier Language Description Metadata Identifier Location CopyRight

17. Some results A good quality mgt project pays off Sometimes automatic metadata gives better results (e.g. format) Unequal spread of low quality metadata element. vCard counts for 68% Only 20 metadata elements are really used LOM.educational has the most diverse practice

18. Quality metrics change over time Total quality as the sum weigthed indicators of quality components Weights are changing  Less on technical integration, human evaluation  More on usefulness of tools for ranking, recommendation, … Reasons: web based resources, advanced tools for improving recall/precision

19. The way to support a purpose is changing From « advanced » search that no one used to facetted search Experience points to the following possible facets:  Provider/Collection  Language  Learning Resource type  Context  Format (aggregated)  Curriculum elements or otherwise: topic, country  Rights

20. Metadata from crowd sourcing & linked data Extending the notion of metadata Paradata LOM

21. Towards new metadata quality From what do humans need towards What do machines need to support humans

22. Thanks! Questions? 22 Interested? frans.vanassche@cs.kuleuven.be