1. SNOMED-CT Vocabulary Standard (Certification) Review Final Recommendations VCDE-WS bi-monthly meeting | 2 Oct 2008 Review Team: Christopher Chute Brian Davis Shantanu Deshpande Kristel Dobratz Hua Min Sal Mungal Mike Riben David Sheretz Stuart Turner Keith Campbell (SNOMED-CT SME/guidance) Thursday, 2 October 2008 Links: GForge Documents: http://tinyurl.com/3vs3vc

2. Criteria Summary Structure  Concept orientation (vagueness, redundancy, ambiguity)  Concept permanence  Non-semantic concept identifiers  Polyhierarchy  Graceful evolution  Formal definitions  Extensibility Content  Content coverage  Polyhierarchy  Rejection of Not Elsewhere Classified Terms  Context representation  Textual definitions* Documentation  Purpose and scope  Statement of intended use  Documentation descriptions  Annotation of versions  Descriptions of tools for acquisition, application and curation Editorial Process  Process for maintenance and extensions  Graceful Evolution  Concept permanence  QA and QC  Organizational governance  Extensions to other terminologies  EVS terminology structure  Access, openness, costs (this star is rising)  Intellectual property/licensing  Community acceptance  External contributions, thoughtful editorial process, etc. * Further discussion

3. Recent Issues: Textual definitions Textual definitions Premise SNOMED-CT inherently does not have textual definitions. A fully specified name, a preferred term and synonyms are not textual definitions For SNOMED-CT, the fully specified name was intended to be a “stand-in” for a textual definition; it is a a unique textual representation, but it does not meet the definition (although perhaps the “spirit”) of a textual definition (or even a fact). It does however serve the purpose of allowing people, with sufficient medical knowledge, to be able to understand its meaning. So although SNOMED does not “provide a clear textual definition of each term in the terminology”, one is able to take the textual representation (fully specified name) combined with description logic to create an adequate description and therefore “partially meets” in this context is a better-fit than “does not meet”. Also, if one places SNOMED in a browser to show hierarchical and role relationships, most would argue that this is sufficient for someone to understand the meaning and the answer (arguably). Outcome Although a similar issue with LOINC was addressed with a “not applicable” assessment, the team felt this was not rigorous enough or appropriate. The team also felt the criteria instrument is currently not sufficient to rigorously and fairly evaluate textual representation for some terminologies (ontologies) such as SNOMED-CT Therefore, a “partially meets”, rather than “not meets” was applied in this case. Further discussion (within and outside scope of SNOMED review) is clearly need regarding impact of textual definitions (descriptions) on EVS and application model annotation, terminology metadata, etc.

4. Discussion Vocabulary evaluation criteria regarding textual definitions are critically important for reliably, consistently and accurately defining the semantics of terms (CDE’s) within an ISO 11179 registry. Concept or knowledge representation by an ontology relies on an approach that does not depend on registries, where manually curated (human) definitions agreed by consensus and formally bound to a concept are used for annotation of various components (classes, attributes, enumerated items in a value domain, etc.) Ontologies do however inherently allow for definitions within a Semantic Web framework (e.g. RDFS or OWL). Concepts (attributes/properties) can be defined by relatively simple relationships (e.g. Object<>Attribute<>Value or Subject<>Predicate<>Object) in RDFS, or.. Via ‘is-a’, partonomy, “isDefinedBy” and many other semantically explicit relationships within an ontology and with additional methods (description logic) to enable reasoners to accurately and reliably, but computationally, derive a “definition” sufficient to annotate a concept. RDFS and OWL (especially) enable interoperability between metadata repositories and global distribution, but making this happen is often not trivial.

5. Discussion Therefore, one dilemma is the need to effectively combine these two disparate, but not incompatible approaches (ISO 11179 registries vs. ontologies). NCI is already doing this well to use vocabularies to instantiate formerly, or through other annotation methods, data element definitions and ontologies within a thesaurus to more richly provide knowledge representation within a domain to further classify this information. SNOMED CT is being used for atomic terms This dilemma is not unique and candidate “vocabularies” on the vocabulary standardization event horizon are structured in a similar way (i.e. ontologies)

6. Discussion Recognized gaps and proposed solutions The current vocabulary evaluation criteria will require revision to become a better evaluation instrument. This includes a fairer evaluation of ontologies but also with consideration of existing tools, infrastructure and workflow requirements within NCI. Revision of this instrument is underway and initial recommendations will be presented to this group for discussion and approval The vocabulary evaluation process will require a staged approach (when appropriate) to “certify” a vocabulary followed by a VCDE consensus effort to further define constraints on usage of this vocabulary within this domain. There will be likely be common approaches as well as need for ad-hoc recommendations based on unique aspects of a given vocabulary. It is recommended that this “second stage” also be vetted through a consensus process by VCDE. This can in part be formally defined and facilitated by the emerging Terminology Metadata Model (schema) Basis of structured documents use as a normative reference populated by a vocabulary owner during the review process as well as companion documents for compatibility reviews to better evaluate intent, scope and usage

7. SNOMED CT Recommendation SNOMED CT has satisfied fully, or in-part all criteria except one Textual definition criteria has been “graded” as partial based on the notion that although textual definitions do not exist, the inherent structure allows for representation via composition or post-coordination and therefore satisfies the “spirit” of these set of criteria. The problem of course, is how to represent these or constrain usage of SNOMED CT within this domain. SNOMED CT already exists and is being used by NCI, is a substantive standard that is being adopted as recommended or required for use by regulatory agencies. Therefore there are objective and subjective aspects here that contribute to this recommendation. Recommendation 1)SNOMED CT be “certified” as an NCI standard vocabulary 2)Usage and implementation be further defined formerly via a consensus approach to VCDE. This will not only satisfy the “second stage” recommendation mentiond previously, but also be the “pilot” of this process 3)Provide recommendations for further enhancing the vocabulary evaluation instrument, especially as a number of ontologies are on the vocabulary standardization roadmap. 4)Discussion by the group….