Natural Language Interfaces to Ontologies Danica Damljanović

University of Sheffield NLP Outline NLIs to ontologies and their usability QuestIO – Question-based Interface to Ontologies Towards better usability using FREyA Demo and evaluation Conclusion 2 10/12/09 Danica Damljanović

University of Sheffield NLP Motivation  Large datasets such as Linked Open Data available  SPARQL/SeRQL:  complex syntax: not easy to learn  writing queries is error-prone task  requires understanding of Semantic Web technologies

University of Sheffield NLP Objective Allow domain experts to query knowledge in RDF/OWL format in a user-friendly manner

University of Sheffield NLP Danica Damljanović Natural Language user-friendly?  (Kaufmann and Bernstein, 2007)‏ Natural Language Interfaces preferred to keywords, menu- guided, and graphical interfaces  (Linckels, 2007): keywords preferred to NL interfaces

University of Sheffield NLP 6 Danica Damljanović NLIs to ontologies

University of Sheffield NLP Error rate caused by… Users not familiar with the covered knowledge Knowledge is not available, but the system is not making that clear to the user i.e. feedback messages not helpful Users have assumptions/misconceptions about the system capabilities and supported language

University of Sheffield NLP 8 Danica Damljanović Usable NLIs to KBs: challenges Robustness Portability What to show? Understanding information need Habitability

University of Sheffield NLP Challenging habitability problem

University of Sheffield NLP Q uestion-based Interface to Ontologies Robust Zero customisation! Easy to use: no training for the user. Deal with incorrectly formulated queries Accept queries of any length and form. Automatically ranks results and shows the highest rank to the user

University of Sheffield NLP NL --> SeRQL query Filtering concepts Ranking concepts Query Creator Query Execution

University of Sheffield NLP An Example compare

University of Sheffield NLP FREyA (Feedback, Refinement, Extended Vocabulary Aggregator) assist the user formulate the query and express his need more precisely Implement user- system interaction and learn from it

University of Sheffield NLP

Example 1 geo:City geo:State new york POC population geo:cityPopulation

University of Sheffield NLP New York is a city

University of Sheffield NLP New York is a state

University of Sheffield NLP POC state area geo:stateArea geo:State geo:isLowestPointOf lowest point Example 2

University of Sheffield NLP Demo

University of Sheffield NLP In lab evaluation Mooney dataset: geography of the United States Relatively small ontology 250 questions require a high level of understanding of semantic meaning Recall and precision: 76% when we identify only the answer type correctly (first POC in the question) Expected: ~91% when we include all POCs from the question and add more refined suggestions (such as max or min for datatype properties of type number)

University of Sheffield NLP Next steps Improvement of the learning mechanism User-based evaluation Trying it with some other datasets

University of Sheffield NLP Thank you! Questions?