1. Smart searches … Smarter metadata Evan Bailey and Sue Carpenter Knowledge Sharing Services

2. Outline Testing of intelligent search  Does it provide an alternative to human-assigned metadata? Background  Our users  The Teaching and Learning Exchange  Our metadata standards  Processes for author-created metadata capture 2/22

3. Our users NSW public school teachers TAFE NSW teachers What we know about what they want  Easy to access online learning resources  Re-usable components  Mapped to NSW curriculum  Like Google 3/22

4. TaLe | Teaching and Learning exchange  Online delivery of learning resources for NSW Department of Education  Schools  TAFE  Parents and Community  Search and browse by learning area and educational level  Easy preview and download 4/22

5. 5/22

6. 6/22

7. 7/22

8. 8/22

9. Our metadata Metadata underpins TaLe Standardisation  Interoperability  DET Learning Resource Metadata  NSW DET vocabularies  Crosswalks 9/22

10. Making metadata invisible Workflow integration  Content creators capture metadata  Easy to use template - The Learning Reference Repository  Automated and default values  QA and training by librarians 10/22

11. The Learning Reference Repository  Easy to use metadata capture template  Workflow integration with resource publishing system Describe and publish resources 11/22

12. 12/22

13. 13/22

14. 14/20

15. Getting buy-in High level of compliance  Easy to use tool for metadata capture  Workflow integration  Quick feedback in TaLe search results  High quality author-created metadata They get it! Metadata matters! Captures subject expert’s knowledge – efficient and effective 15/22

16. The promise of the new Intelligent search engines  Make sense of unstructured information  Conceptual and contextual understanding  Related concepts Improved efficiency?  Let the machine do it! Eduction  Generation of metadata values by the search engine 16/22

17. Proof of concept testing Can search engine eduction provide a satisfactory alternative to human- assigned metadata on Curriculum Support site, in terms of assigning metadata values for stages of learning and key learning areas? Test environment  Curriculum Support resources (24,000)  Test cases  Comparison with human-assigned metadata  Review of search engine-assigned values 17/22

18. Results Search engine eduction alone not reliable enough  Did not assign values where it should have – false negatives 30%  Assigned values where it should not have – false positives 25% Users would lose confidence in search Time and resources needed to train the engine Aid human-assigned metadata – not a replacement 18/22

19. Characteristics of educational resources  Student resources  Context not always explicit  Trigger terms not present 19/22

20. The value of evidence Strengths of the existing process  Content experts = Expert taggers  Easy capture tool  Metadata standard  Specific terminology for NSW curriculum Metadata adds value 20/22

21. What’s next Watching brief and more testing  Intelligent search with intranet content  Implement related concepts  Social tagging 21/22

22. More … for later TaLe http://www.tale.edu.auhttp://www.tale.edu.au Who are we?  The Centre for Learning Innovation http://www.cli.nsw.edu.au/cli/index.shtm The Centre for Learning Innovation http://www.cli.nsw.edu.au/cli/index.shtm  Knowledge Sharing Services http://www.cli.nsw.edu.au/cli/focus/standards.shtm Knowledge Sharing Services http://www.cli.nsw.edu.au/cli/focus/standards.shtm  evan.bailey@det.nsw.edu.au evan.bailey@det.nsw.edu.au  sue.carpenter@det.nsw.edu.au sue.carpenter@det.nsw.edu.au Our metadata profile  The DETLRM The DETLRM 22/22