1. Triplestore Experiences Nathan Wilhelmi 11/27/2012 NCAR - CISL/TDD/VETS

2. Our Experiences… Disclaimers: ▫Did not have an ontologist ▫Codebase passed through multiple developers Timelines (changing landscape) ▫Started work 2006 ▫Stopped active development ~2011 ▫Sesame version 2.3.0

3. Why a Triplestore? Search functionality ▫Faceted ▫Free text Model metadata ▫Metadata storage ▫Display Semantic web

4. Initial Architecture Authoritative metadata source was RDBMS Metadata harvested into the triplestore at periodic intervals Triplestore only contained metadata to drive search Sesame used as a stand alone service

5. Sesame Triplestore Standalone Sesame server ▫Stability problems ▫No security, triplestore could be updated by anyone Changed to in-memory store ▫Stable ▫Picked up performance improvements Embedded triplestore was only internally referencing ▫RDF didn’t work outside of the application ▫Distilled to key-value store

6. Internal Referencing........

7. Performance For our query patterns were not seeing needed performance Inferencing was removed and performance improved to acceptable levels for <5k datasets ▫ Target volume 50K datasets Sparql missing key operators: ordering, limits

8. Tooling Support Managing the triplestore ▫Protégé round trips didn’t work well ▫Dump full triple store to XML and grep by hand Deleting and updating triples ▫Deletes were difficult, dangling triples ▫Rebuild from authoritative sources

9. Implementation Issues Schema-less design was perceived as faster ▫Rapid ontology changes during development ▫Still needed data migration tools Modeling the problem domain ▫Modeled a triplestore, not the domain ▫Very tightly coupled code was difficult to maintain and replace Steep learning curve for new developers

11. URIs Are Foundational Properly encoding URIs ▫Created unencoded URIs within the triplestore ▫Queries were created with string concentration ▫Lead to broken queries and data Generated instance URIs through a lossy algorithm to get around encoding ▫Could only relate from source -> triple store

12. Our Current Path Forward Using SOLR Search ▫Fantastic search tool! Metadata in RDBMS ▫Working well ▫Effective tools, including schema migration ▫Scales very well for our metadata Still needed to expose RDF metadata…

13. RDF with RDBMS resource://${gateway.name?upper_case}#${dataset.persistentIdentifier} ${dataset.name}

14. Looking Forward Storing metadata ▫Content management systems? ▫NoSql storage options? Modeling complicated relationships ▫Neo4J looks promising…

15. Questions / Discussion Nathan Wilhelmi ▫wilhelmi@ucar.edu