1. Natural Language Interfaces to Ontologies Danica Damljanović danica@dcs.shef.ac.uk

2. University of Sheffield NLP Outline NLIs to ontologies and their usability QuestIO – Question-based Interface to Ontologies Towards better usability using FREyA Demo and evaluation Conclusion 2 10/12/09 Danica Damljanović

3. University of Sheffield NLP Motivation  Large datasets such as Linked Open Data available  SPARQL/SeRQL:  complex syntax: not easy to learn  writing queries is error-prone task  requires understanding of Semantic Web technologies

4. University of Sheffield NLP Objective Allow domain experts to query knowledge in RDF/OWL format in a user-friendly manner

5. University of Sheffield NLP Danica Damljanović Natural Language user-friendly?  (Kaufmann and Bernstein, 2007)‏ Natural Language Interfaces preferred to keywords, menu- guided, and graphical interfaces  (Linckels, 2007): keywords preferred to NL interfaces

6. University of Sheffield NLP 6 Danica Damljanović NLIs to ontologies

7. University of Sheffield NLP Error rate caused by… Users not familiar with the covered knowledge Knowledge is not available, but the system is not making that clear to the user i.e. feedback messages not helpful Users have assumptions/misconceptions about the system capabilities and supported language

8. University of Sheffield NLP 8 Danica Damljanović Usable NLIs to KBs: challenges Robustness Portability What to show? Understanding information need Habitability

9. University of Sheffield NLP Challenging habitability problem

10. University of Sheffield NLP Q uestion-based Interface to Ontologies Robust Zero customisation! Easy to use: no training for the user. Deal with incorrectly formulated queries Accept queries of any length and form. Automatically ranks results and shows the highest rank to the user

11. University of Sheffield NLP NL --> SeRQL query Filtering concepts Ranking concepts Query Creator Query Execution

12. University of Sheffield NLP An Example 1.15 1.19 compare

13. University of Sheffield NLP FREyA (Feedback, Refinement, Extended Vocabulary Aggregator) assist the user formulate the query and express his need more precisely Implement user- system interaction and learn from it

14. University of Sheffield NLP

15. Example 1 geo:City geo:State new york POC population geo:cityPopulation

16. University of Sheffield NLP New York is a city

17. University of Sheffield NLP New York is a state

18. University of Sheffield NLP POC state area geo:stateArea geo:State geo:isLowestPointOf lowest point Example 2

19. University of Sheffield NLP Demo http://gate.ac.uk/freya

20. University of Sheffield NLP In lab evaluation Mooney dataset: geography of the United States Relatively small ontology 250 questions require a high level of understanding of semantic meaning Recall and precision: 76% when we identify only the answer type correctly (first POC in the question) Expected: ~91% when we include all POCs from the question and add more refined suggestions (such as max or min for datatype properties of type number)

21. University of Sheffield NLP Next steps Improvement of the learning mechanism User-based evaluation Trying it with some other datasets

22. University of Sheffield NLP Thank you! Questions?