1. Value Domain and Pick List Support in LexEVS 5.1 Sridhar Dwarkanath Mayo Clinic Dwarkanath.sridhar@mayo.edu CaBIG Architecture/VCD Joint Workspace F2F Atlanta, GA USA October 2009

2. Session Goal Familiarize with Value Domain Definition and Pick List Definition part of LexGrid model and Their functions supported in LexEVS 5.1 release.

3. LexEVS – Lexical Enterprise Vocabulary Services LexEVS Developed to allow federated, open source system for delivering vocabulary content. Strategic EVS interface, replacing the legacy EVS API and the EVS 3.2 model. Key infrastructure tool for caBIG ®. Available through caCORE distribution. Represents the next generation of NCI Enterprise Vocabulary Services.

4. LexEVS – Lexical Enterprise Vocabulary Services LexEVS combines LexGrid and LexEVS Java API LexGrid : A standard model of controlled vocabularies and ontologies. Defines how vocabularies should be formatted and represented programmatically. Rich enough to represent vocabularies provided in numerous source formats including: Open Biomedical Ontologies (OBO) Web Ontology Language (OWL), e.g., NCI Thesaurus Unified Medical Language System (UMLS) Rich Release Format (RRF), e.g., NCI MetaThesaurus LexEVS Java API : Applies LexGrid vision and technologies to requirements of the caBIG® community. Provide a flexible implementation for vocabulary storage and persistence. Provide standard tooling for load and distribution of vocabulary content.

5. LexEVS – Lexical Enterprise Vocabulary Services LexEVS includes the following components: Java API - A Java interface based on the LexGrid Object Model Distributed LexBIG (DLB) API - A Java interface based on the LexGrid data model and relies on a LexEVS Proxy and Distributed LexEVS Adapter to provide remote clients access to the native LexEVS API REST/HTTP Interface - Offers an HTTP based query mechanism. Results are returned in either XML or HTML formats SOAP/Web Services Interface - Provides a programming language neutral Service- Oriented Architecture (SOA) LexEVS 5.0 Grid Service - An interface which uses the caGRID infrastructure to provide access to the native LexEVS API via the caGRID Services

6. LexEVS – Lexical Enterprise Vocabulary Services LexEVS Environment Architecture Diagram showing architecture of LexEVS. At the bottom, as Data Source, we have lexGrid MySQL database. Core API layer on top of data source layer contains LexGrid Model Objects, Java APU and the Lucene Index Files. Application Service layer on top of Core API, contain Distributed Java API and LexEVS CaGrid API. And at the top, we have Client layer, with Client Application which could include Java QBE (Query By Example), Web/Grid Service (Soap/HTTP/Rest) and Java RMI. Model Objects LexEVS caCORE SDK APIs Java API LexGrid MySQL DB Lucene Index Files Distributed Java LexEVS caGrid API Java (QBE) Application Service Client Web/Grid Service (Soap/HTTP/Rest) Java (RMI) ( Distributed) Client Application Core API Data Source RMI

7. LexGrid Model : Value Domain Definition Diagram containing Value Domain part of LexGrid Logical Model. Value Domain Definition : URI should be unique Versionable DefinitionEntry 0 or more definitionEntry Each definitionEntry can reference either code system or value domain or can contain entityReference Multiple definitionEntries can be combined using local operators ‘OR’, ‘SUBTRACT’ or ‘AND’ entityReference entityCode is the start concept and a required field. Other fields are for rules. URI should be unique Versionable 0 or more definitionEntry Each definitionEntry can reference either code system or value domain or can contain entityReference entityCode is start concept referenceAssociation Applicable rules : leafOnly targetToSource transitiveClosure DefinitionEntries can be combined using logical operators ‘OR’, ‘SUBTRACT’, or ‘AND’

8. Value Domain Definitions Possible Forms Code system/concept code + relationship + additional rules (leaf only, immediate children,...) Code system – all concept codes in the code system Other Value Domain – all resolved concept codes returned by referencing Value Domain Code system/concept code – individual code Combination of any of the above with or/and/difference operators

9. Value Domain Definition Resolution A value domain definition has to be made against a specific version of a code system. But it doesn’t have to be resolved against the same version. Even a simple list (a, b, c, d) needs to be resolved as, at some future date, “c” might be retired. Resolution does not create static artifact.

10. LexGrid Model : Pick List Definition Diagram showing Pick List part of LexGrid Logical Model. PickListId unique Value Domain URI required completeDomain – True/False True – Dynamic, ignores pickListEntryNode False – Static, default 0 or More pickListEntryNode Each pickListEntryNode can contain either have pickListEntry or pickListEntryExclusion PickListEntry : pickText and entityCode are required fields. PickListId unique Value domain URI required completeDomain – True/False True – Dynamic, ignores pickListEntryNode. False – Static, default versionable 0 or more pickListEntryNode Each pickListEntryNode can contain either pickListEntry or pickListEntryExclustion versionable pickText and entityCode are required fields.

11. Pick List Definitions Possible Forms Value Domain Definition – all concept codes in the value domain completeDomain = true Dynamic form Code system/concept code – individual code (inclusion and exclusion) completeDomain = false Subset of Value Domain Static form

12. Pick List Resolution A Pick List definition has to be made against a specific value domain. Even a simple list (a, b, c, d) needs to be resolved as, at some future date, “c” might be retired. Resolution does not create static artifact.

13. LexEVS Value Domain Services Admin Services loadValueDomain(InputStream, boolean) - Loads value domain using inputStream. loadValueDomain(String, boolean) - Loads value domain by reading XML file. loadValueDomain(ValueDomainDefinition, String) - Loads supplied valueDomainDefinition object. removeAllValueDomains() - Removes all value domain definitions from the system. removeValueDomain(URI) - Removes supplied value domain definition from the system. validate(URI, int) - Perform validation of the value domain definition without loading data. dropValueDomainTables() - Drops value domain tables only if there are no value domain and pick list entries.

14. LexEVS Value Domain Services Query Services getCodingSchemesInValueDomain(URI) - Returns list of coding scheme summary that is referenced by the supplied value domain. getValueDomainDefinition(URI) - Returns value domain definition for supplied value domain URI. getValueDomainEntitiesForTerm(String, URI, String) - Resolves the value domain supplied and restricts to the term and matchAlgorith supplied. listValueDomains(String) - Return the URI's for the value domain definition(s) for the supplied domain name. If the name is null, returns everything. If the name is not null, returns the value domain(s) that have the assigned name. resolveValueDomain(URI, AbsoluteCodingSchemeVersionReferenceList) - Resolve a value domain using the supplied set of coding scheme versions.

15. LexEVS Value Domain Services Query Services (Cont…) isConceptInDomain(String, URI) - Determine if the supplied entity code is a valid result for the supplied value domain and, if it is, return the particular codingSchemeVersion that was used. isConceptInDomain(String, URI, CodingSchemeVersionOrTag, URI) - Determine if the supplied entity code is a valid result for the supplied domain. isDomain(String, String, CodingSchemeVersionOrTag) - Determine if the supplied entity code is of type valueDomain in supplied coding scheme and, if it is, return the true, otherwise return false. isSubDomain(URI, URI) - Check whether childValueDomainURI is a child of parentValueDomainURI.

16. LexEVS Pick List Services Admin Services loadPickList(InputStream, boolean) - Loads pick list using inputStream. loadPickList(PickListDefinition, URI) - Loads supplied PickListDefinition object. loadPickList(String, boolean) - Loads pick list by reading XML file location supplied. removePickList(String) - Removes pick list definition from the system that matches supplied pickListId. validate(URI, int) - Perform validation of the picklist definition without loading data.

17. LexEVS Pick List Services Query Services getPickListDefinitionById(String) - Returns pickList definition for supplied pickListId getPickListDefinitionsForDomain(URI) - Returns all the pickList definitions that represents supplied valueDomain URI. getPickListValueDomain(String) - Returns an URI of the represented valueDomain of the pickList. listPickListIds() - Returns list of pickListIds that are available in the system. resolvePickList(String, boolean) - Resolves pickList definition for supplied pickListId. resolvePickListForTerm(String, String, String, String, String[], boolean) - Resolves pickList definition for supplied arguments - Term to restrict, match algorithm to use, language to restrict, list of context to restrict, sort flag (If True, will be sorted by text in ascending order.)

18. LexEVS 5.0 vs LexEVS 5.1 LexEVS 5.0 Release Current Release Value Domain and Pick List services available as an extension LexEVS 5.1 Release November 2009 Integrated within LexEVS API

19. Value Domain and Pick List Services in LexEVS Both Value Domain and Pick List Services will be available in following components of LexEVS 5.1 : Java API - A Java interface based on the LexGrid 5.0 Object Model Distributed LexBIG (DLB) API - A Java interface based on the LexGrid 2009/01 data model and relies on a LexEVS Proxy and Distributed LexEVS Adapter to provide remote clients access to the native LexEVS API

20. Value Domain and Pick List Services in LexEVS Value Domain and Pick List Services available in LexEVS Environment Diagram showing architecture of LexEVS. At the bottom, as Data Source, we have lexGrid MySQL database. Core API layer on top of data source layer contains LexGrid Model Objects, Java APU and the Lucene Index Files. Application Service layer on top of Core API, contain Distributed Java API and LexEVS CaGrid API. And at the top, we have Client layer, with Client Application which could include Java QBE (Query By Example), Web/Grid Service (Soap/HTTP/Rest) and Java RMI. Model Objects LexEVS caCORE SDK APIs Java API LexGrid MySQL DB Lucene Index Files Distributed Java LexEVS caGrid API Java (QBE) Application Service Client Web/Grid Service (Soap/HTTP/Rest) Java (RMI) ( Distributed) Client Application Core API Data Source RMI

21. Future Plans Future plans for Value Domain and Pick List in LexEVS Persist resolved data. Include Value Domain and Pick List as part of following LexEVS components : REST/HTTP Interface - Offers an HTTP based query mechanism. Results are returned in either XML or HTML formats SOAP/Web Services Interface - Provides a programming language neutral Service- Oriented Architecture (SOA) LexEVS Grid Service - An interface which uses the caGRID infrastructure to provide access to the native LexEVS API via the caGRID Services

22. More Information caBIG® Vocabulary Knowledge Center : https://cabig-kc.nci.nih.gov/Vocab/KC/index.php/LexBig_and_LexEVS Download Software Documentation and Training Presentations and Demos LexGrid and LexBIG UML Diagram LexGrid and LexBIG XML Schema Sample Database caBIG LexEVS Developer Boot Camp : November 17 th, 18 th, and 19 th, 2009

23. Questions ?