1. LexBIG/EVS API Overview NCBO Seminar Series October 2008

2. What is LexBIG? A specific API to serve vocabularies based on the LexGrid model, developed for the Cancer Biomedical Informatics Grid (caBIG®) community. Open source implementation. Underpins current NCI Enterprise Vocabulary Services (EVS). Represents the next generation EVS API. Additional contributions and tuning to meet the needs of the NCBO community, based on development of NCBO BioPortal services.

3. Other OBO Content Export Import OWL XML OBO Text Protégé RRF XML Representation LexGrid Vocabulary Model Data Repository LexBIG Architecture Tools And Services Access Programming Interfaces APIs LexBIG/ EVS … CTS Apps Java Embed Web/Grid Services Distributed API Application Server Each LexGrid ‘Node’ provides the software, metadata, indexes, and backing data store to service one or more vocabularies. Each LexBIG Installation represents one LexGrid Node and Java API to administer and query data. File System: Metadata & Indexes

4. LexBIG/EVS caGrid Service LexBIG/EVS Distributed Service LexBIG/EVS Java API Conceptual Overview LexGrid Model & Storage Browsers and Applications

5. LexBIG/EVS Java API LexBIG/EVS API LexGrid Model & Storage

6. LexBIG/EVS API – Characteristics API coverage Administrative FunctionsAdministrative Functions Query Code Systems and Available MetadataQuery Code Systems and Available Metadata Query Concepts, Concept Properties, and QualificationsQuery Concepts, Concept Properties, and Qualifications Query Concept Relationships and QualificationsQuery Concept Relationships and Qualifications API characteristics Conscious separation of service and data classesConscious separation of service and data classes Deferred query resolutionDeferred query resolution Payload optimizationPayload optimization Support iterationSupport iteration Defined extension points (loaders, exporters, sort algorithms, filters, convenience methods)Defined extension points (loaders, exporters, sort algorithms, filters, convenience methods)

7. LexBIG/EVS API – Example Prerequisites ICD-9-CM loaded from UMLS distribution (RRF source files) LoadUMLSFiles [-s null] [-mf uri] [-v int] [-lp uri] [-nf] -in uri [-t id] [-mh int] [-a] Example: LoadUMLSFiles -in file:///path/to/directory/ -s “ICD9CM" -nf -afile:///path/to/directory/ -s,--sources Comma-delimited list of source vocabularies to load. -mh,--mrhier Process hierarchical relationships stored by the MRHIER file. 0=none, 1=HCD-tagged items (default) -a,--activate ActivateScheme on successful load. -in,--input URI or path specifying the directory containing the NLM files. -lp,--loadPrefs URI or path specifying location of the Loader Preferences file. -mf,--manifest URI or path specifying location of the manifest file. -nf,--noFail If specified, indicates that processing should not stop for recoverable errors. -t,--tag An optional tag ID (e.g. 'PRODUCTION' or 'TEST') to assign. -v,--validate Validation only; no load. If specified, 'nf', 'a' and 't‘ are ignored. 0 to verify files exist.

8. LexBIG/EVS API – Example Let’s query some concepts … Target a code system Define a new LexBIGService as starting point. Define a concept ‘space’ (a codedNodeSet) for ICD-9-CM, version 2007 Note: Initially unrestricted and unresolved LexBIGService lbs = new LexBIGServiceImpl(); CodedNodeSet cns = lbs.getCodingSchemeConcepts("ICD-9-CM", null);

9. LexBIG/EVS API – Example Prerequisites ICD-9-CM loaded from UMLS distribution (RRF source files) Target a code system Define a concept ‘space’ (a codedNodeSet) for ICD-9-CM, version 2007 Initially unrestricted and unresolved Restrict the space based by adding constraints Property ‘Semantic Type’ -> exact match -> “Disease or Syndrome” LexBIGService lbs = new LexBIGServiceImpl(); CodedNodeSet cns = lbs.getCodingSchemeConcepts("ICD-9-CM", null); cns.restrictToMatchingProperties( Constructors.createLocalNameList( "Semantic_Type"), null, "Disease or Syndrome", “exactMatch", null);

10. LexBIG/EVS API – Example Prerequisites ICD-9-CM loaded from UMLS distribution (RRF source files) Target a code system Define a concept ‘space’ (a codedNodeSet) for ICD-9-CM, version 2007 Initially unrestricted and unresolved Restrict the space based by adding constraints Property ‘Semantic Type’ -> exact match -> “Disease or Syndrome” Primary text match -> sounds like -> ‘infeksion’ LexBIGService lbs = new LexBIGServiceImpl(); CodedNodeSet cns = lbs.getCodingSchemeConcepts("ICD-9-CM", null); cns.restrictToMatchingProperties( Constructors.createLocalNameList( "Semantic_Type"), null, "Disease or Syndrome", “exactMatch", null); cns.restrictToMatchingDesignations("infeksion", SearchDesignationOption. PREFERRED_ONLY, “DoubleMetaphoneLuceneQuery”, null);

11. LexBIG/EVS API – Example Prerequisites ICD-9-CM loaded from UMLS distribution (RRF source files) Target a code system Define a concept ‘space’ (a codedNodeSet) for ICD-9-CM, version 2007 Initially unrestricted and unresolved Restrict the space based by adding constraints Property ‘Semantic Type’ -> exact match -> “Disease or Syndrome” Primary text match -> sounds like -> ‘infeksion’ Any text stemmed match -> ‘classify’ (to match ‘classified’, ‘classifying’, etc) LexBIGService lbs = new LexBIGServiceImpl(); CodedNodeSet cns = lbs.getCodingSchemeConcepts("ICD-9-CM", null); cns.restrictToMatchingProperties( Constructors.createLocalNameList( "Semantic_Type"), null, "Disease or Syndrome", “exactMatch", null); cns.restrictToMatchingDesignations("infeksion", SearchDesignationOption. PREFERRED_ONLY, “DoubleMetaphoneLuceneQuery”, null); cns.restrictToMatchingDesignations("classify", SearchDesignationOption.ALL, "StemmedLuceneQuery“, null);

12. LexBIG/EVS API – Example Prerequisites ICD-9-CM loaded from UMLS distribution (RRF source files) Target a code system Define a concept ‘space’ (a codedNodeSet) for ICD-9-CM, version 2007 Initially unrestricted and unresolved Restrict the space based by adding constraints Property ‘Semantic Type’ -> exact match -> “Disease or Syndrome” Primary text match -> sounds like -> ‘infeksion’ Any text stemmed match -> ‘classify’ (to match ‘classified’, ‘classifying’, etc) Concept must be active LexBIGService lbs = new LexBIGServiceImpl(); CodedNodeSet cns = lbs.getCodingSchemeConcepts("ICD-9-CM", null); cns.restrictToMatchingProperties( Constructors.createLocalNameList( "Semantic_Type"), null, "Disease or Syndrome", “exactMatch", null); cns.restrictToMatchingDesignations("infeksion", SearchDesignationOption. PREFERRED_ONLY, “DoubleMetaphoneLuceneQuery”, null); cns.restrictToMatchingDesignations("classify", SearchDesignationOption.ALL, "StemmedLuceneQuery“, null); cns.restrictToStatus(ActiveOption.ACTIVE_ONLY, null);

13. LexBIG/EVS API – Example Prerequisites ICD-9-CM loaded from UMLS distribution (RRF source files) Target a code system Define a concept ‘space’ (a codedNodeSet) for ICD-9-CM, version 2007 Initially unrestricted and unresolved Restrict the space based by adding constraints Property ‘Semantic Type’ -> exact match -> “Disease or Syndrome” Primary text match -> sounds like -> ‘infeksion’ Any text stemmed match -> ‘classify’ (to match ‘classified’, ‘classifying’, etc) Must contain a property with name -> ‘UMLS_CUI’ Concept must be active Indicate sort preferences and limit number returned Sort by code, ascending Limit to top 5 … cns.restrictToStatus(ActiveOption.ACTIVE_ONLY, null); ResolvedConceptReferenceList list = cns.resolveToList( Constructors.createSortOptionList(new String[] { "matchToQuery", "code" }), null, null, 5);

14. LexBIG/EVS API – Example Prerequisites ICD-9-CM loaded from UMLS distribution (RRF source files) Target a code system Define a concept ‘space’ (a codedNodeSet) for ICD-9-CM, version 2007 Initially unrestricted and unresolved Restrict the space based by adding constraints Property ‘Semantic Type’ -> exact match -> “Disease or Syndrome” Primary text match -> sounds like -> ‘infeksion’ Any text stemmed match -> ‘classify’ (to match ‘classified’, ‘classifying’, etc) Must contain a property with name -> ‘UMLS_CUI’ Concept must be active Indicate sort preferences and limit number returned Sort by code, ascending Limit to top 5 Resolve!

15. LexBIG/EVS API – Example 2 OK, now find some relationships… Target a code system Define an unrestricted graph for a target ontology (e.g. ICD-9-CM) CodedNodeGraph cng = lbs.getNodeGraph("ICD-9-CM", null, null);

16. LexBIG/EVS API – Example 2 OK, now find some relationships… Target a code system Define an unrestricted graph for a target ontology (e.g. ICD-9-CM) Restrict by adding constraints Restrict to hierarchical relationships (UMLS-defined ‘CHD’ = has child) CodedNodeGraph cng = lbs.getNodeGraph("ICD-9-CM", null, null); cng.restrictToAssociations(Constructors.createNameAndValueList(“CHD"), null);

17. LexBIG/EVS API – Example 2 OK, now find some relationships… Target a code system Define an unrestricted graph for a target ontology (e.g. ICD-9-CM) Restrict by adding constraints Restrict to hierarchical relationships (UMLS-defined ‘CHD’ = has child) Indicate extent of navigation Set focal point (008 = ‘Intestinal infections due to other organisms’) Maximum 2 levels, full reporting at all levels Moving only in forward direction No application of filter extensions, etc Maximum 50 nodes resolved overall CodedNodeGraph cng = lbs.getNodeGraph("ICD-9-CM", null, null); cng.restrictToAssociations(Constructors.createNameAndValueList(“CHD"), null); cng.resolveAsList( Constructors.createConceptReference("008", "ICD-9-CM"), true, false, 2, 2, null, null, null, null, 50);

18. LexBIG/EVS API – Example 2 OK, now find some relationships… Target a code system Define an unrestricted graph for a target ontology (e.g. ICD-9-CM) Restrict by adding constraints Restrict to parent/child relationships (UMLS-defined ‘PAR’ = has parent) Restrict to the codedNodeSet defined in the previous example Indicate extent of navigation Set focal point (008 = ‘Intestinal infections due to other organisms’) Maximum 2 levels, moving in forward direction Maximum 50 nodes resolved overall Resolve!

19. LexBIG/EVS Distributed Service LexBIG/EVS Java API LexBIG API - Distributed LexGrid Model & Storage

20. Database Server LexBIG/EVS API – Distributed Distributed LexBIG Server RMI LexBIG on Local System LexBIG Install Database Server LexBIG Install JDBC Direct Distributed LexBIG API Proxy Client System LexBIG Client Proxy

21. LexBIG/EVS API – Distributed // Set the URL (for distributed access) and establish a remote service … String evsUrl = “http://evsapi.nci.nih.gov/evsapi41/http/remoteService”; LexBIGService lbSvc = EVSApplicationService.getRemoteInstance(evsUrl); // Additional code is unchanged … CodedNodeSet cns = lbs.getCodingSchemeConcepts("ICD-9-CM", null); cns.restrictToMatchingProperties( Constructors.createLocalNameList( "Semantic_Type"), null, "Disease or Syndrome", “exactMatch", null); cns.restrictToMatchingDesignations("infeksion", SearchDesignationOption. PREFERRED_ONLY, “DoubleMetaphoneLuceneQuery”, null); cns.restrictToMatchingDesignations("classify", SearchDesignationOption.ALL, "StemmedLuceneQuery“, null); … Same API functions; requires change to service declaration only Does not expose administration options

22. LexBIG/EVS caGrid Service LexBIG/EVS Distributed Service LexBIG/EVS Java API LexBIG/EVS API - caGrid LexGrid Model & Storage

23. Database Server LexBIG/EVS API - caGrid Distributed LexBIG Server RMI LexBIG on Local System LexBIG Install Database Server LexBIG Install JDBC Direct Distributed Database Server LexBIG API Proxy Client System caGrid Host Server Client System Distributed LexBIG Server RMI LexBIG Install Grid JDBC TCP LexBIG Proxy LexBIG Client Proxy

24. Same API Functionality ClientcaGrid ServiceDistributed LexBIG Calls caGrid ‘getSupportedCodingSchemes()’ Calls Distributed LexBIG ‘getSupportedCodingSchemes() Returns result of call to caGrid Service Results are returned to client with all appropriate caGrid security mechanisms LexBIG/EVS API - caGrid

25. caCORE SDK APIs LexGrid Model & Storage

26. Query-by-example (QBE) system Based on LexGrid model Java Language Binding Web Services (SOAP) Web Services (REST HTTP / XML) caCORE EVS Server Web Services XML / HTML Java QBE LexBIG Install DAO Cache Service Layer Database Server JDBC caCORE SDK APIs Hibernate

27. caCORE SDK APIs - Example SDK-Generated Query by Example User Interface:

28. caCORE SDK APIs - Example Example Restful Request (HTML)… Example Restful Output (HTML shown, XML also available)…

29. LexBIG/EVS caGrid Service LexBIG/EVS Distributed Service LexBIG/EVS Java API Browsers and Applications LexGrid Model & Storage Browsers and Applications

30. User Interfaces - LexBIG GUI

31. User Interfaces - BioPortal

32. Project Links LexEVS Project http://gforge.nci.nih.gov/projects/lexevs/ caBIG® Vocabulary Knowledge Center https://cabig-kc.nci.nih.gov/Vocab/KC/index.php/Main_Page