1. http://userlab.open.ac.uk/ Andrew Brasher a.j.brasher@open.ac.uk Andrew Brasher, Patrick McAndrew Userlab, IET, Open University Human-Generated Learning Object Metadata

2. 2 Summary Examine factors affecting quality of human generated metadata How can ontologies and systems which exploit them help? Conclusions

3. 3 Quality of human generated metadata Often neither complete nor consistent Factors which influence the quality of human produced metadata : –Motivation of the producers –Accuracy –Consistency Currier, S., et al (2004)Chan, L.M (1989)

4. 4 Socio-cognitive engineering Sharples et al. (2002)

5. 5 In this talk……. An ontology is a formal explicit specification of a shared conceptualisation (Gruber 1993) but…. Issues with langauge dependence (Guarino 1998)

6. 6 An aim of metadata To deliver right resources, to right people, in the right place, at the right time to aid these people achieve their learning goals Content Collaborative environment People

7. 7 Implications To deliver appropriate learning objects implies descriptions of –Temporal, spatial context –Social context (including descriptions of people) –Learning objects –Devices –Delivery infrastructure and processes to select and deliver the right resources

8. 8 For generation of LO metadata: Intrinsic Extrinsic 2 sources: contained within the resource itself, are a necessary part of the resource itself. E.g.: format of a resource, and title of a resource. not contained within the resource itself. E.g.: personal or community views about the expected use of the resource.

9. 9 Generation of metadata Metadata Source Process Intrinsic sources. E.g.: format of a resource, and title of a resource. Extrinsic sources. E.g.: personal or community views about the expected use of the resource. Metadata? Process?

10. 10 In practice, there are two types of sources which require human intervention at point of production to create metadata descriptors: extrinsic sources and most intrinsic sources within non-textual resources. Human intervention e.g. expected use e.g. sound, movies, multimedia

11. 11 1)Interactivity Type; 2)Learning Resource Type; 3)Interactivity Level; 4)Semantic Density; 5)Intended End User Role; 6)Context; 7)Typical Age Range; 8)Difficulty; 9)Typical Learning Time; 10)Description; 11)Language. Most intrinsic sources within non-textual resources Extrinsic sources e.g. expected use Educational category of the IEEE Learning Object Metadata 1)Interactivity Type; 2)Learning Resource Type; 3)Interactivity Level; 4)Semantic Density; 5)Intended End User Role; 6)Context; 7)Typical Age Range; 8)Difficulty; 9)Typical Learning Time; 10)Description; 11)Language. 2 sources requiring human generation of metadata

12. 12 Example Difficulty “How hard it is to work with or through this learning object for the typical intended target audience.” “NOTE—The “typical target audience” can be characterized by data elements 5.6:Educational.Context and 5.7:Educational.TypicalAgeRange.” very easy easy medium difficult very difficult IEEE, (2002).

13. 13 Example: “Learning in the connected economy”

14. 14 Example: “Learning in the connected economy”

15. 15 How to create difficulty meatadata? By authors using structured vocabularies Task analysis

16. 16 Metadata creation Kabel et al., 2003VDEX, 2004

17. 17 Example LO

18. 18 Example

19. 19 Categorisation “Typical learning time” / hours“Difficulty” < 1Very easy 1 <= “Typical learning time” < 3Easy 3 <= “Typical learning time” < 4Medium “Typical learning time” > 4Difficult

20. 20 Multiple Contexts Previous idea works for a single context What about multiple contexts? –Who? –How could it be exploited?

21. 21 Goal To enable a system to compare the difficulty of LO’s on the same topic –designed for different ‘target audiences’ Looking for a general ‘solution’

22. 22 context2 (museum) Metadata museum Metadata degree context1 (Art History degree) easy Target audience relationship LO degreeLO museum

23. 23 Art history degree course team create difficulty metadata Museum guide team create difficulty metadata Relationship metadata created by ‘managers’ Metadata Creation Automatic generation? Task analysis, endemic motivation Task analysis, endemic motivation

24. 24 context2 (museum) Metadata museum Metadata degree context1 (Art History degree) easy Target audience relationship LO degreeLO museum ContextRelationship class The ContextRelationship class describes relationships between assignments of difficulty in different contexts. An example of how data contained within an instance should be interpreted is: "For students in the Museum context, the assignments of difficulty made for students in the Art History degree context are more difficult."

25. 25 Ontology Assignments of difficulty made in this context context1 are perceived as “more difficult” than assignments of difficulty made in this context context2 Assignments of difficulty made in this context context1 are perceived as “less difficult” than assignments of difficulty made in this context context2 Assignments of difficulty made in this context context1 are perceived as “as difficult” as assignments of difficulty made in this context context2

26. 26 Multiple contexts context2 (museum) context1 (Art History degree) easy Possibility of disagreements? Possibility of refinements? “More difficult than” “Less difficult than” context3 (Secondary school) easy

27. 27 Conclusions Consider and exploit endemic motivation in the system design Single context: LOM schema sufficient(?) Multiple contexts: more complex

28. 28 Future plans More complex representation needed –Ontology / OWL Other domains –User Profiles Task model Metadata creation task model - Ontologies are one of the many tools that can be used Forget this: it’s never going to work!

29. 29