Smart searches … Smarter metadata Evan Bailey and Sue Carpenter Knowledge Sharing Services

Outline Testing of intelligent search  Does it provide an alternative to human-assigned metadata? Background  Our users  The Teaching and Learning Exchange  Our metadata standards  Processes for author-created metadata capture 2/22

Our users NSW public school teachers TAFE NSW teachers What we know about what they want  Easy to access online learning resources  Re-usable components  Mapped to NSW curriculum  Like Google 3/22

TaLe | Teaching and Learning exchange  Online delivery of learning resources for NSW Department of Education  Schools  TAFE  Parents and Community  Search and browse by learning area and educational level  Easy preview and download 4/22

5/22

6/22

7/22

8/22

Our metadata Metadata underpins TaLe Standardisation  Interoperability  DET Learning Resource Metadata  NSW DET vocabularies  Crosswalks 9/22

Making metadata invisible Workflow integration  Content creators capture metadata  Easy to use template - The Learning Reference Repository  Automated and default values  QA and training by librarians 10/22

The Learning Reference Repository  Easy to use metadata capture template  Workflow integration with resource publishing system Describe and publish resources 11/22

12/22

13/22

14/20

Getting buy-in High level of compliance  Easy to use tool for metadata capture  Workflow integration  Quick feedback in TaLe search results  High quality author-created metadata They get it! Metadata matters! Captures subject expert’s knowledge – efficient and effective 15/22

The promise of the new Intelligent search engines  Make sense of unstructured information  Conceptual and contextual understanding  Related concepts Improved efficiency?  Let the machine do it! Eduction  Generation of metadata values by the search engine 16/22

Proof of concept testing Can search engine eduction provide a satisfactory alternative to human- assigned metadata on Curriculum Support site, in terms of assigning metadata values for stages of learning and key learning areas? Test environment  Curriculum Support resources (24,000)  Test cases  Comparison with human-assigned metadata  Review of search engine-assigned values 17/22

Results Search engine eduction alone not reliable enough  Did not assign values where it should have – false negatives 30%  Assigned values where it should not have – false positives 25% Users would lose confidence in search Time and resources needed to train the engine Aid human-assigned metadata – not a replacement 18/22

Characteristics of educational resources  Student resources  Context not always explicit  Trigger terms not present 19/22

The value of evidence Strengths of the existing process  Content experts = Expert taggers  Easy capture tool  Metadata standard  Specific terminology for NSW curriculum Metadata adds value 20/22

What’s next Watching brief and more testing  Intelligent search with intranet content  Implement related concepts  Social tagging 21/22

More … for later TaLe Who are we?  The Centre for Learning Innovation The Centre for Learning Innovation  Knowledge Sharing Services Knowledge Sharing Services   Our metadata profile  The DETLRM The DETLRM 22/22