1. Linked Data Best Practices (and Abuses) Lessons Learned in IBM Rational Arthur Ryman 2014-04-15

2. Best Practices Publishing vocabularies Data model customization Real-world things JSON and RDF Multi-valued and optional properties Provenance and inverse properties Ontologies and constraints 2

3. PUBLISHING VOCABULARIES 3

4. Publishing vocabularies We should use established vocabularies if they exist – W3C, Dublin Core, OSLC, … Any new terms we define should be described in vocabulary documents rooted at http://jazz.net/nshttp://jazz.net/ns – propose generally useful terms to OSLC When you look up an RDF term, you should get its vocabulary document – HTML for web browsers – RDF for programs, e.g. query builders – e.g. http://jazz.net/ns/qm/rqm#Categoryhttp://jazz.net/ns/qm/rqm#Category 4

5. Vocabulary page for http://jazz.net/ns/qm/rqm#Category 5

6. How to publish a vocabulary We have a new public wiki! – https://jazz.net/wiki/bin/view/LinkedData https://jazz.net/wiki/bin/view/LinkedData Read the guidelinesguidelines Create a wiki page and attach the HTML, Turtle, and RDF/XML files Request a review from Nelson – Allow dev time to address issues Arthur will redirect jazz.net/ns to the wiki 6

7. LinkedData wiki 7

8. Abuses You published your vocabulary but skimped on the content – e.g. minimal or cryptic comments You published your vocabulary, but didn’t keep it up-to-date – e.g. Focal Point 227292227292 You created some new terms but didn’t publish your vocabulary – e.g. JLIP Tracked Resource Set 306919306919 8

9. DATA MODEL CUSTOMIZATION 9

10. Data model customization Many of our tools allow customization – e.g. RTC work items We need to expose the custom data elements as RDF Tools should allow users to map custom data elements to externally defined RDF terms – industry standards – corporate standards When no mapping is specified, tools should generate local RDF terms and vocabularies – vocabularies are needed by query authors – tools must host the vocabularies they generate 10

11. Abuses Your tool generates a cryptic URI for local RDF terms – Obfuscates meaning – Forces humans to access vocabulary document Your tool does not generate a vocabulary document for local RDF terms – e.g. RTC 304143304143 – see following case study When the mapping to RDF is changed, your tool does not create TRS change events for just the affected resources 11

12. Case study: RTC Work Items Some attributes are built-in Some are defined by OSLC CM 2.0 Some are user defined Consider Priority 12

13. Project area editor allows customization 13

14. Enumerated values should specify RDF URIs (External Value) 14

15. Priority values are enumerated 15

16. Get the resource URL 16

17. Look for priority in the RDF representation of Task 224727 17

18. RDF triple for priority Subject (good) https://jazzop05.rtp.raleigh.ibm.com:9943/jazz/ resource/itemName/com.ibm.team.workitem.Wo rkItem/224727 Predicate (bad) http://open-services.net/ns/cm-x#priority Object (ugly) https://jazzop05.rtp.raleigh.ibm.com:9943/jazz/ oslc/enumerations/_QYx2UBIzEd6bpunPP4ZLOA/ priority/priority.literal.l3 18

19. Object of priority is not an RDF vocabulary term 19

20. Problems The priority predicate comes from a non-existent vocabulary (bad) – http://open-services.net/ns/cm-x# – RDF vocabularies should be dereferenceable – OSLC should publish it, tagged as archaicarchaic The object is a dereferenceable URI (good), but not a vocabulary term (ugly) – Need rdfs:label, rdfs:comment for query authors Result: no easy way to write queries based on priority 20

21. Best Practice for external vocabularies RTC project template should refer to external vocabularies for standard terms – OSLC CM V3 defines priority and 4 values Teach and enable clients to create corporate standard vocabularies for reuse of common terms (UA) – Needed for cross-project queries Provide export/import UI to manage vocabularies – E.g. Focal Point uses simple spreadsheet format 21

22. Best Practice for local vocabularies RTC (and all other tools) should generate a local RDF vocabulary for all user-defined terms – Include rdfs:label, rdfs:comment for query authors (and other consumers) LQE admin should load user-defined vocabularies into LQE to make them available to queries – provide programmatic integration, e.g. a special purpose vocabulary TRS 22

23. Best Practice for all vocabularies When an administrator changes the RDF representation of a set of resources, corresponding change events MUST be added to the TRS change log – Add/remove custom attributes and values – Modify mapping to RDF URIs Allow the administrator to make multiple representation changes and then manually trigger the generation of change events – Batch multiple representation changes together to minimize re-indexing time and server load 23

24. REAL-WORLD THINGS 24

25. La Trahison des Images "The famous pipe. How people reproached me for it! And yet, could you stuff my pipe? No, it's just a representation, is it not? So if I had written on my picture "This is a pipe", I'd have been lying!“ - René MagritteRené Magritte 25

26. Real-world things Linked Data differentiates between two kinds of thing – Information, e.g. a document on the web – Real-world, e.g. a person Both kinds should be identified with HTTP URIs Looking up a real-world URI should result in an information resource that contains information about the real-world thing – URI-references (hash URIs) – HTTP redirect: 303 See Other (303 URIs) Refer to Cool URIs for the Semantic WebCool URIs for the Semantic Web 26

27. Example foaf:Person Suppose you create a document, http://people.org/johnsmith, about John Smith on 2013-09-17 http://people.org/johnsmith The following is nonsense because John Smith was not created on 2013-09-17: a foaf:Person. dcterms:created “2013-09-17”^^xsd:date. The following makes sense: a foaf:Person. dcterms:created “2013-09-17”^^xsd:date. 27

28. Abuses Failure to differentiate between a person and an account owned by a person – Leads to nonsense triples – Focal Point Defect 234212 Focal Point Defect 234212 – JTS Defect 307861 JTS Defect 307861 – See following JTS users case study NOTE: email address is the preferred way to identify people across tools 28

29. Work items refer to people 29

30. JTS Users OSLC Core specifies that the object of dcterms:creator, dcterms:contributor, oslc:modifiedBy should be a resource of class foaf:Agent or foaf:Person (real-world) RTC implements OSLC CM and has triples like: dcterms: creator, dcterms:contributor. 30

31. RDF representation of person contains nonsense 31

32. Best Practice The property j.1:archived applies to the user account (information resource), not the person (real-world) Solution 1: use hash URIs for people: Solution 2: use 303 URIs for accounts (preferred by Philippe): 32

33. 303 URI Solution @prefix foaf:. @prefix jfs:. a foaf:OnlineAccount, jfs:archived false. a foaf:Person; foaf:account, foaf:img ; foaf:mbox ; foaf:name "Arthur Ryman"; foaf:nick "ryman". 33

34. JSON AND RDF 34

35. JSON Familiar to OO and Web developers Popularity fueled by Cloud e.g. Amazon uses JSON as the payload in AWS REST APIs as an alternative to SOAP and XML – Simpler/faster to handle by web clients Use is spreading across the stack – MongoDB, CouchDB/Cloudant – node.js 35

36. JSON and RDF Some developers are saying: “JSON is simpler and more popular than RDF. Let’s use JSON instead of RDF.” – This is a false dichotomy JSON is just as problematic as XML for data integration – JSON and XML are message formats Linked Data is our integration strategy – RDF expresses semantics Use JSON-LD, now a W3C standardJSON-LD – OSLC and Rational should publish standard contexts See following LQE Security Context case studyLQE Security Context case study 36

37. Initial JSON design Simple, but no explicit semantics Use of UUIDs instead of HTTP URIs [ { "security_context_id" : "urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6", "name" : "Resources for Alpha project" }, { "security_context_id" : "urn:uuid:g92e5gbf-8efd-22e1-b876-11b1d02f7cg7", "name" : "Resources for Beta project" } ] 37

38. Equivalent JSON-LD design { "@context": { "@base": "https://example.com/sc",https://example.com/sc "dcterms": "http://purl.org/dc/terms/"http://purl.org/dc/terms/ }, "@graph": [ { "@id": "#1", "dcterms:title": "Resources for Alpha project" }, { "@id": "#2", "dcterms:title": "Resources for Beta project" } ] } 38

39. Final JSON-LD design with type info { "@graph": [ { "@id": "https://example.com/sc",https://example.com/sc "@type": "http://open-services.net/ns/core/sc#SecurityContextList"http://open-services.net/ns/core/sc#SecurityContextList }, { "@id": "https://example.com/sc#1",https://example.com/sc#1 "@type": "http://open-services.net/ns/core/sc#SecurityContext",http://open-services.net/ns/core/sc#SecurityContext "http://purl.org/dc/terms/title": "Resources for Alpha project"http://purl.org/dc/terms/title }, { "@id": "https://example.com/sc#2",https://example.com/sc#2 "@type": "http://open-services.net/ns/core/sc#SecurityContext",http://open-services.net/ns/core/sc#SecurityContext "http://purl.org/dc/terms/title": "Resources for Beta project"http://purl.org/dc/terms/title } ] } 39

40. MULTI-VALUED AND OPTIONAL PROPERTIES 40

41. Multi-valued and optional properties RDF documentations contain sets of triples Model multi-valued properties by a set of triples that share a common subject and object Model the absence of an optional property by an empty set of triples 41

42. Abuses Model multiple values of a property by concatenating the values into a single object – Defeats database indexing – Slows queries since substring matching must be used Model the absence of an optional value using the presence of an empty string – Adds many unnecessary triples – Slows queries (longer scans) – Sometimes an empty string is a meaning value – Sometimes an empty string is lexically invalid See following RTC tag case study Defect 271867Defect 271867 42

43. “Tags” is multi-valued “Estimate” is optional 43

44. RDF representation @prefix dcterms:. @prefix rtc_cm:. @prefix xsd:. @base. dcterms:subject "datagap, oslc, next_release_candidate, data_gap, reporting-gap"^^xsd:string ; … rtc_cm:estimate ""^^xsd:long. Syntax validated OK. There were warnings: Typed literal has an invalid lexical value: Input string was not in the correct format: s.Length==0.: ""^^. 44

45. PROVENANCE AND INVERSE PROPERTIES 45

46. Provenance: Where did the triple come from? A statement is represented by a triple Triples from multiple documents may be merged and queried – Default graph is a triple store When storing RDF documents, the document URL is often used as the name of a graph (e.g. in LQE) – triple + graph name = quad – triple stores are really quad stores Provenance of triples is important in several use cases – Updating a document – Access control – VVC (which version) 46

47. Provenance and authority The authority (trust) of a triple depends on the author of the document that contains the triple Triples should be placed in the document that the author is authorized to modify – When creating a link from A to B, put the link in the document that the author is editing, not necessarily A or B or both – Document C may contain links from A to B 47

48. Inverse properties Directed relations between resources (links) may be stated in two equivalent ways, e.g. – Testcase1 validates Requirement2. – Requirement2 isValidatedBy Testcase1. There is no benefit to having mutual inverse pairs of properties The existence of mutual inverse pairs of properties makes query authoring more complex, and query execution more expensive A triple should be put in the document that the author of the triple is editing (provenance) – There is no special significance attached to being the subject of a triple See OSLC guidance on preferred direction of propertiesOSLC guidance – Direction should be from downstream to upstream, – e.g. test case validates requirement 48

49. Abuses OSLC domain specs define many pairs of mutual inverse predicates Recommendation – Deprecate one member of each pair – Replace deprecated property in all RDF representations and queries 49

50. ONTOLOGIES AND CONSTRAINTS 50

51. Vocabularies and Ontologies A vocabulary defines the meaning of terms – Use RDFS: rdfs:label, rdfs:comment, rdfs:isDefinedBy, … An ontology defines inference rules – Given a set of triples, infer more triples – Use RDFS: rdfs:domain, rdfs:range, rdfs:subClassOf, … – Use OWL for more complex inference rules 51

52. Ontologies and Constraints Ontologies are not designed to define integrity constraints – See Linked Data Interfaces for examplesLinked Data Interfaces An RDFS or OWL reasoner will add triples to create a model for the ontology A reasoner will report an inconsistency if it cannot create a model – However, this mechanism cannot in practice be used to check for typical integrity constraints 52

53. Best Practice: Ontologies Your triples may end up in a reasoner one day, so only add inference rules when they produce the intended results If you define generic properties, such as “uses”, then you probably SHOULD NOT define rdfs:domain and rdfs:range If you define type-specific properties, such as “usesTestCase” then rdfs:domain and rdfs:range MAY make sense e.g. If you intend to infer that the object of oslc_qm:usesTestCase is an oslc_qm:TestCase then include the following triple in an ontology: oslc_qm:usesTestCase rdfs:range oslc_qm:TestCase. 53

54. Best Practice: Constraints W3C is starting an activity on RDF validation – See W3C workshopW3C workshop We have submitted the OSLC Resource Shape specification to W3C – See Resource Shape 2.0Resource Shape 2.0 Use Resource Shape 2.0 to describe integrity constraints on RDF documents 54

55. Other topics Blank nodes – Mean there exists or some – use fragment ids for internal resources Containers – Avoid Seq, Bag, List – Use Linked Data Platform containersLinked Data Platform Consuming external vocabularies – Tools should gracefully degrade when external resources are unreachable – Be a well-behaved HTTP client wrt caching, etc. 55