1. LexEVS in a Local Environment Interacting with LexEVS 5.0 November 2009

2. Interacting with LexEVS 5.0 Course Goals When you complete this course, you will be able to: Discuss the LexEVS API in a local environment Install and configure LexEVS for the local environment Perform local code implementation exercises Utilize LexEVS loader technology Demonstrate the use of the lbGUI for loading and code implementation

3. Interacting with LexEVS 5.0 Lesson Syllabus Lesson 1: Overview of Local Environment Identify the components for a local installation Lesson 2: Java API Review Core Services: Identify the basic interfaces and entry points into LexEVS. Service Extensions: Identify how to register an extension, using extension interfaces, using the loaders and LexBIG service convenience methods. Loaders: Identify loaders available and utilize to install content. Convenience Methods: Identify how the convenience methods are used to make the API more useful. LexEVS GUI: Use the GUI to demonstrate API functionality. Lesson 3: Hands-on setup Set up LexEVS 5.0 for the local environment. Lesson 4: Hands-on exercises: Using Eclipse Code Exercises

4. Interacting with LexEVS 5.0 Lesson Syllabus Lesson 5: Using the Loaders Using loader scripts with optional manifest and preferences files Lesson 6: Using the lbGUI Ask and answer questions about how your implementation may work before actually writing any code.

5. Lesson 1: Overview of Local Environment When you complete this lesson, you will be able to: Identify the software stack required for LexEVS local installation Understand the options for LexEVS backend storage Locate the download artifacts produced for various LexEVS components List components shipped with LexEVS local runtime installer Recognize third party jars that are not included with LexEVS but are required for it’s operation.

6. Lesson 1: Overview of Local Environment Components Reference software stack: Java 1.5 MySQL 5.0.45 LexEVS 5.0 local runtime installer Software stack should already be in place on your machine With the exception of local runtime installer LexEVS installer provides a self contained installation of LexEVS and required dependencies One exception- Driver for MySQL does not ship with LexEVS due to license restrictions **We’ll show you how to deal with this issue as the installation proceeds**

7. Lesson 1: Overview of Local Environment Alternate Storage Options db2 PostgreSQL Oracle Hypersonic

8. Lesson 1: Overview of Local Environment Download Options A wide range of user components are available as alternatives from the Vocabulary Knowledge Center download site. https://cabig-kc.nci.nih.gov/Vocab/KC/index.php/LexEVS_5.0_Components

9. Lesson 1: Overview of Local Environment Download Options Included in these resources are: Separate LexEVS runtime and dependency jars LexEVS source Documentation for system administrators and developers Javadocs Release notes Example code Resources for other installs we’ll be using in the distributed and grid environments

10. Lesson 1: Overview of Local Environment Installer Jar What’s included: example code Admin command line shell scripts and wrappers JUnit test suite Includes ant script – allows users to immediately test installed applications Sample terminologies Javadocs Admin and developer documentation Configuration files Release notes Expert level GUI interface to most LexEVS methods

11. Lesson 1: Overview of Local Environment Installer Jar What’s not included: MySQL JDBC driver (required for this exercise) Configuration details for your system XML configuration file for command line installation

12. Lesson 1: Overview of Local Environment Review  Where can one find the download components for LexEVS 5.0?

13. Lesson 1: Overview of Local Environment Answer  Where can one find the download components for LexEVS 5.0? The Vocabulary Knowledge Center web page

14. Lesson 2: Java API Review When you complete this lesson you will be able to: Identify basic interfaces and entry points into LexEVS Describe how extensions are created registered in LexEVS Discuss use of loaders and LexBIG service convenience methods Identify loaders available Describe the purpose of the lbGUI

15. Lesson 2: Java API Review Core Services This slide shows the LexBIG Service as a UML class – listing all of its methods with a call out indicating it is the entry point into LexEVS Initial entry point into LexEVS

16. Lesson 2 : Java API Review CodedNodeSet This Slide shows how the user can get other interfaces like the coded node set by calling methods on the entry point – in this case getCodingSchemeConcepts() which can return a reference to a set of concepts to the user.

17. Lesson 2 : Java API Review LexBIGServiceManager This slide shows how the service manager can be accessed from the service entry point using the getserviceManager() method.

18. Lesson 2 : Java API Review LexBIGServiceMetadata Here we see how the LexBIGService can provide the user with an interface capable of browsing external metadata files for a given coding scheme.

19. Lesson 2 : Java API Review HistoryService A history service interface can also be retrieved from the LexBIG service object using the getHistoryService Method.

20. Lesson 2 : Java API Review Service Extensions Pertinent extensions: Loaders Convenience Methods Generic Extensions Some special extensions are registered inside a Resource Manager class: Example: LexEVS Loaders and Convenience Methods Others extensions are registered using an xml file to hold values that will populate an Extension Description object New/User created extensions, must implement a LexEVS extension class

21. Lesson 2 : Java API Review Loaders Loaders are implemented as extensions, but effectively are a part of the code base Shell Scripts and example code for implementing the current loader set into an application are shipped with the installer package Can be used as standalone applications in the LexEVS install User does not need to create their own application to load content Current list of loaders includes those for the following formats: RRF OWL HL7-RIM database OBO Radlex using Protégé Frames

22. Lesson 2 : Java API Review Convenience Methods ** Remember: LexBIGServiceConvenienceMethods is a type of Generic Extension. It’s important not to think of if as an Extension Interface.** Wrapper LexEVS API calls which: Simplify what can be complex LexEVS code calls Narrow the scope of an otherwise flexible api Useful for developers seeking examples for complex code calls Example: recursive node graph traversal

23. Lesson 2 : Java API Review LexEVS GUI GUI demonstrates API Functionality The GUI is an expert level tool, intended to allow the user or developer to exercise API level calls in a convenient graphical mode.

24. Lesson 2: Java API Review Review  What is the initial entry point into LexEVS?

25. Lesson 2: Java API Review Answer  What is the initial entry point into LexEVS? LexBIG Service

26. Lesson 2: Java API Review Review  What do we call applications that extend the plug-in interface to LexEVS?

27. Lesson 2: Java API Review Answer  What do we call applications that extend the plug-in interface to LexEVS? Extensions

28. Lesson 2: Java API Review Review  Is the LexEVS GUI intended as an end user tool?

29. Lesson 2: Java API Review Answer  Is the LexEVS GUI intended as an end user tool? The lbGUI is a prototype designed as a testing tool for the expert and developer. It’s not intended to be an end user tool.

30. Lesson 2: Java API Review Review  Characterize the convenience method interface.

31. Lesson 2: Java API Review Answer  Characterize the convenience method interface. These methods are code wrappers generally intended for specific purposes. As such, the may be a useful guide for developer implementation, but may need some adjustment to insure they return value specific to the users purpose.

32. Lesson 3: Hands-on Setup When you complete this lesson you will be able to: Install an instance of LexEVS local runtime Configure that instance to run against a local MySQL data base

33. Lesson 3: Hands-on Setup Installation Instructions Navigate to LexEVS bootcamp\LexEVS bootcamp day 2\resources on your desktop. Refer to the word document titled installsetuplexevs.doc This document is the installation instructions for the local environment. We will be using this to complete the exercises.

34. Lesson 3 Hands-on Setup Quick Review 1.Launch the LexEVS Installation Program 2.Select the Installation Language 3.Start the Installation 4.Review the LexEVS Readme 5.Accept the User Agreement 6.Set the installation Path 7.Accept default component selection 8.The program will now install 9.Click ‘Done’ upon completion 10. Edit lbconfig.props file to adjust MySQL username/password 11. Copy the MySQL driver to the runtime/sqlDrivers folder 12. Copy the sample-manifest.xml file to the test/resources/testData folder 13.. Load content

35. Lesson 4: Hands-on Exercises When you complete this lesson you will be able to: Utilize the Eclipse IDE Demonstrate a basic understanding of Eclipse utilization in order to complete the exercises Instantiate a LexEVS based terminology service Access other service interfaces through this service Retrieve code sets and graphs from the service Verbalize how to use restrictions upon those interfaces Acquire iterators and lists using resolve methods Employ built in extensions of LexEVS Create and use extensions of LexEVS

36. Lesson 4 : Hands-on Exercises Using Eclipse Navigate to LexEVS bootcamp\LexEVS bootcamp day 2\resources on your desktop Open word document titled usingEclipse We’ll follow the word document throughout these exercises. This is an introduction to the Eclipse Environment. This establishes a rule of thumb that will work well for the most of these environments The extension project will require an additional ant build step and won’t use JUnits exclusively for exercise execution

37. Lesson 4 : Hands-on Exercises Exercise Categories The Hands-on code examples are broken down into four categories: 1.Examples 2.Exercises 3.Solutions 4.Sandbox We will be evaluating code coverage in these categories: Core services Extensions Convenience Methods

38. Lesson 4 : Hands-on Exercises Non-Coding Exercises Other non-coding exercises: Loaders (Implemented as extensions) The LexEVS GUI Let’s begin…

39. Lesson 4: Hands-on Exercises Example 1 Coded Node Set Examples: Navigate to lexevs.bootcamp.local.examples.lexbigservice Getting a coded node set – open for editing GetLexBIGService.java Notice that the LexBIGService instantiates using a method call instead of a constructor call. This method insures the user will always call the same LexBIGService object. If you like you can right click and run the example as a java application, but this method doesn’t really do anything other than return a valid LexBIGService object.

40. Lesson 4: Hands-on Exercises Example 1a Coded Node Set Examples: Navigate to lexevs.bootcamp.local.examples.lexbigservice Getting a coded node set – open for editing GetSupportedCodingSchemes.java This method will return a list of coding schemes from the Terminology service, giving the user an initial reference to what exists there and how it might be accessed. Right click and run the example as a java application so you can see what coding schemes are currently loaded.

41. Lesson 4: Hands-on Exercises Exercise 1 Now navigate to the lexevs.bootcamp.local.exercises.lexbigservice package and modify the class GetLexBIGServiceExercise and method getLocalLexBIGService() so that it also instantiates LexBIGServiceImpl. Next open for editing the class GetSupportedCodingSchemesExercise.java and modify the getCodingSchemeRenderingList() method so that it gets a list of coding schemes from the LexBIGService. In all cases remove the line throwing the exception inside these methods throw new ExerciseNotCompleteException(); When finished right click on the class in the package explorer and choose run as -> JUnit from the resulting menu.

42. Lesson 4: Hands-on Exercises Solutions 1 Now navigate to the lexevs.bootcamp.local.solutions.lexbigservice package and open the file GetLexBIGServiceExerciseSolution.java. Check your work in the solutions packages matching the exercises.

43. Lesson 4: Hands-on Exercises Sandbox 1 Now navigate to lexevs.bootcamp.local.sandbox.lexbigservice and open LexBIGServiceSandbox.java. It’s time to play a bit with LexBIGService. If you are familiar with Eclipse auto complete then you can use it to prompt for any number of possible method calls. Type the instantiation of the LexBIGService class “lbs” and then type a “.” afterwards. Eclipse should bring up a list of methods available to the user from the LexBIGService interface. Try some of them out.

44. Lesson 4: Hands-on Exercises Exercise Framework This establishes the pattern for the exercises. 1.We’ll look at the example 2.Do the exercise 3.Check the solution 4.Play in the sandbox

45. Lesson 4: Hands-on Exercises Example 2: Coded Node Set Coded Node Set Examples: Navigate to lexevs.bootcamp.local.examples.codednodeset Getting a coded node set – open for editing GetCodedNodeSet.java This demonstrates how to get a reference to all concepts in the coding scheme. This reference will be manipulated in ensuing restriction calls to tailor it to the needs of the end user. If you like you can right click and run the example as a java application. In this case the print utility should show all concepts in the coding scheme, but this isn’t what you’d normally want to do.

46. Lesson 4: Hands-on Exercises Exercise 2: Coded Node Set Navigate to: lexevs.bootcamp.local.exercises.codednodeset Open the class GetCodedNodeSetExercise and review the code. Modify the method private CodedNodeSet getCodedNodeSet(LexBIGService lbs) so that it gets a reference to the coded nodes in the sample OWL terminology. When finished, right click on the class in the package explorer and choose Run as -> JUnit Test from the resulting menu. Review the JUnit results in the JUnit window to ensure successful completion The solution is provided at: lexevs.bootcamp.local.solutions.codednodeset. GetCodedNodeSetExerciseSolution

47. Lesson 4: Hands-on Exercises Example 3: Coded Node Set Iterator Coded Node Set Examples: Navigate to lexevs.bootcamp.local.examples.codednodeset Getting a resolved concept reference iterator– open for editing GetCodedNodeSetIterator. This demonstrates how to get an iterator over all concepts references when a coded node set is resolved. Iterators can be powerful performance enhancing tools, since they get references to Lucene indexed values yet do no resolving of values until the user decides to do so. If you like you can right click and run the example as a java application.

48. Lesson 4: Hands-on Exercises Exercise 3: Coded Node Set Iterator Navigate to: lexevs.bootcamp.local.exercises.codednodeset Open the class ResolveCodedNodeSetIteratorExercise and review the code. Modify the method protected ResolvedConceptReferencesIterator getResolvedConceptReferencesIterator() so that it gets an iterator over a collection of resolved concept references from the sample OWL terminology. When finished, right click on the class in the package explorer and choose Run as -> JUnit Test from the resulting menu. Review the JUnit results in the JUnit window to ensure successful completion The solution is provided at: lexevs.bootcamp.local.solutions.codednodeset. ResolveCodedNodeSetIteratorExerciseSolution

49. Lesson 4: Hands-on Exercises Example 4: Coded Node Set List Coded Node Set Examples: Navigate to lexevs.bootcamp.local.examples.codednodeset Getting a ResolvedConceptReferenceList– open for editing ResolveCodedNodeSetList.java. This demonstrates how to get List of Resolved Concept References when a coded node set resolveToList method is called. Resolving to a list of concept references gives the user the opportunity to structure the set of returned objects by volume and list size. If you like you can right click and run the example as a java application.

50. Lesson 4: Hands-on Exercises Exercise 4: Coded Node Set List Navigate to: lexevs.bootcamp.local.exercises.codednodeset Open the class ResolveCodedNodeSetListExercise and review the code. Modify the method getResolvedConceptReferenceList(CodedNodeSet cns) So that it resolves a concept reference list from the sample OWL terminology. When finished, right click on the class in the package explorer and choose Run as -> JUnit Test from the resulting menu. Review the JUnit results in the JUnit window to ensure successful completion The solution is provided at: lexevs.bootcamp.local.solutions.codednodeset. ResolveCodedNodeSetListExerciseSolution

51. Lesson 4: Hands-on Exercises Example 5: Resolved Concept References List Coded Node Set Examples: Navigate to lexevs.bootcamp.local.examples.codednodeset Getting a ResolvedConceptReferenceList– open for editing ResolveCodedNodeSetList.java This demonstrates how to get List of Resolved Concept References when a coded node set resolveToList method is called. Resolving to a list of concept references gives the user the opportunity to structure the set of returned objects by volume and list size. If you like you can right click and run the example as a java application.

52. Lesson 4: Hands-on Exercises Exercise 5: Resolved Concept References List Navigate to: lexevs.bootcamp.local.exercises.codednodeset Open the class ResolveCodedNodeSetListExercise and review the code. Modify the method getResolvedConceptReferenceList(CodedNodeSet cns) So that it resolves a concept reference list from the sample OWL terminology. When finished, right click on the class in the package explorer and choose Run as -> JUnit Test from the resulting menu. Review the JUnit results in the JUnit window to ensure successful completion The solution is provided at: lexevs.bootcamp.local.solutions.codednodeset. ResolveCodedNodeSetListExerciseSolution

53. Lesson 4: Hands-on Exercises Example 6: List Restricted by Concept Coded Node Set Examples: Navigate to lexevs.bootcamp.local.examples.codednodeset Getting a ResolvedConceptReferenceList restricted by a concept code – open for editing RestrictCodedNodeSetByCode.java This shows how a restriction can pinpoint a concept in a coding scheme – even a large one, when the concept code is known. If you like you can right click and run the example as a java application.

54. Lesson 4: Hands-on Exercises Exercise 6: List Restricted by Concept Navigate to: lexevs.bootcamp.local.exercises.codednodeset Open the class RestrictCodedNodeSetByCodeExercise and review the code. Modify the method addRestriction(CodedNodeSet cns) So that it resolves a concept reference list from the sample OWL terminology given a single code as the restriction. When finished, right click on the class in the package explorer and choose Run as -> JUnit Test from the resulting menu. Review the JUnit results in the JUnit window to ensure successful completion The solution is provided at: lexevs.bootcamp.local.solutions.codednodeset. RestrictCodedNodeSetByCodeExerciseSolution

55. Lesson 4: Hands-on Exercises Example 7: List Restricted by Presentation Coded Node Set Examples: Navigate to lexevs.bootcamp.local.examples.codednodeset Getting a ResolvedConceptReferenceList restricted by a presentation text search – open for editing RestrictCodedNodeSetByDesignation.java This shows how a restriction can allow a broad search of properties known as presentations which are similar (Some would say synonymous) in meaning to the preferred presentation for a given concept using a given text match. "heart", //the text to match SearchDesignationOption.ALL, //whether to search all designation, only Preferred or only Non-Preferred "contains", //the match algorithm to use null //the language to match (null matches all) If you like you can right click and run the example as a java application.

56. Lesson 4: Hands-on Exercises Exercise 7: List Restricted by Presentation Navigate to: lexevs.bootcamp.local.exercises.codednodeset Open the class RestrictCodedNodeSetByDesignationExercise and review the code. Modify the method addRestriction(CodedNodeSet cns) So that it resolves a concept reference list from the sample OWL terminology given a text match search on the presentations of the contained concepts. When finished, right click on the class in the package explorer and choose Run as -> JUnit Test from the resulting menu. Review the JUnit results in the JUnit window to ensure successful completion The solution is provided at: lexevs.bootcamp.local.solutions.codednodeset. RestrictCodedNodeSetByDesignationExerciseSolution

57. Lesson 4: Hands-on Exercises Example 8: List Restricted by Property Coded Node Set Examples: Navigate to lexevs.bootcamp.local.examples.codednodeset Getting a ResolvedConceptReferenceList restricted by a property name and text search – open for editing RestrictCodedNodeSetByNamedProperty.java This shows how a restriction can name a property in the coding scheme and search on text in that property. cns.restrictToMatchingProperties( Constructors.createLocalNameList("code"), //the Property Name to match null, //the Property Type to match (null matches all) "C48589", //the text to match "exactMatch", //the match algorithm to use null //the language to match (null matches all) If you like you can right click and run the example as a java application.

58. Lesson 4: Hands-on Exercises Exercise 8: List Restricted by Property Navigate to: lexevs.bootcamp.local.exercises.codednodeset Open the class RestrictCodedNodeSetByNamedPropertyExercise and review the code. Modify the method restrictCodedNodeSetByNamedProperty() So that it resolves a concept reference list from the sample OWL terminology given a text match search on the named property of the contained concepts. When finished, right click on the class in the package explorer and choose Run as -> JUnit Test from the resulting menu. Review the JUnit results in the JUnit window to ensure successful completion The solution is provided at: lexevs.bootcamp.local.solutions.codednodeset. RestrictCodedNodeSetByNamedPropertyExerciseSolution

59. Lesson 4: Hands-on Exercises Self Study Exercises: Coded Node Set Finish any coded node set exercises

60. Lesson 4: Hands-on Exercises Self Study Answers When you have completed all the exercises run them as JUnit(s)

61. Lesson 4: Hands-on Exercises Coded Node Set Sandbox Coded node set Sandbox Suggestions: Experiment with the public API of the LexBIGService calling such methods as those listing extensions, matching algorithms, etc Retrieve an iterator over all the objects in the sample coding scheme coded node set and experiment with the iterator API including next(int), scroll(int) and getNext(int, int) Call a resolveAsList method on the CodedNodeSet that has more parameters and experiment with setting parameters such as “boolean resolveObjects”

62. Lesson 4: Hands-on Exercises Example 9: Coded Node Graph Coded Node Graph Examples: Navigate to lexevs.bootcamp.local.examples.codednodegraph Getting a ResolvedConceptReferenceList for a node graph – open for editing getCodedNodeGraph.java This shows how to get another kind of reference into the coding scheme. This reference is capable of retrieving concept codes and their relationships to one another. lbs.getNodeGraph( BootcampConstants.SAMPLE_ONTOLOGY_NAME, //The name of the ontology to resolve null, //The ontology version ('null' will return the version marked 'PRODUCTION', //or disregard the version if there is only one ontology with that name loaded. null //The relationship 'container' name to resolve -- 'null' resolves all. If you like you can right click and run the example as a java application.

63. Lesson 4: Hands-on Exercises Exercise 9: Coded Node Graph Navigate to: lexevs.bootcamp.local.exercises.codednodegraph Open the class GetCodedNodeGraphExercise and review the code. Modify the method getCodedNodeGraph() So that it gets a node graph from the terminology service. We have the print utility set up so that it resolves the top nodes only from the sample terminology. When finished, right click on the class in the package explorer and choose Run as -> JUnit Test from the resulting menu. Review the JUnit results in the JUnit window to ensure successful completion The solution is provided at: lexevs.bootcamp.local.solutions.codednodegraph. GetCodedNodeGraphExerciseSolution

64. Lesson 4: Hands-on Exercises Example 10: Coded Node Graph Coded Node Graph Examples: Navigate to lexevs.bootcamp.local.examples.codednodegraph Getting a ResolvedConceptReferenceList based on the “focus” or root of the graph – open for editing GraphFocus.java Continued …

65. Lesson 4: Hands-on Exercises Example 10: Coded Node Graph Supposing you don’t want an entire graph structure, one way to get a portion of the graph is to select a node and resolve relationships to or from that node. Here is the resolveAsList parameter set. focus, //The graph focus code false, //resolve forward true, //resolve backward 1, //depth (levels) of associations to resolve 1, //depth (levels) of coded entries to resolve null, //PropertyNames to resolve (null resolves all) null, //PropertyTypes to resolve (null resolves all) null, //sorts -1 //cap the amount of results returned (-1 means return all results) If you like you can right click and run the example as a java application.

66. Lesson 4: Hands-on Exercises Exercise 10: Coded Node Graph Navigate to: lexevs.bootcamp.local.exercises.codednodegraph Open the class GraphFocusExercise and review the code. Modify the method resolveCodedNodeGraph() So that it gets a node graph from the terminology service. We have the print utility set up so that it resolves the top nodes only from the sample terminology. When finished, right click on the class in the package explorer and choose Run as -> JUnit Test from the resulting menu. Review the JUnit results in the JUnit window to ensure successful completion The solution is provided at: lexevs.bootcamp.local.solutions.codednodegraph. GraphFocusExerciseSolution

67. Lesson 4: Hands-on Exercises Example 11: Graph by Associations Coded Node Graph Examples: Navigate to lexevs.bootcamp.local.examples.codednodegraph Getting a ResolvedConceptReferenceList for a node graph – open for editing RestrictCodedNodeGraphByAssociation.java Restricting a graph by it’s associations is a basic way to limit what kind of results returned to the user. cng.restrictToAssociations( Constructors.createNameAndValueList("subClassOf"), //limit the graph to only specified Associations null //match Association Qualifiers (null matches all) ); If you like you can right click and run the example as a java application

68. Lesson 4: Hands-on Exercises Exercise 11: Graph by Associations Navigate to: lexevs.bootcamp.local.exercises.codednodegraph Open the class RestrictCodedNodeGraphByAssociationExercise and review the code. Modify the method addRestrictions() so that it restricts the resulting ResolvedConceptReferenceList to a single type of association. When finished, right click on the class in the package explorer and choose Run as -> JUnit Test from the resulting menu. Review the JUnit results in the JUnit window to ensure successful completion The solution is provided at: lexevs.bootcamp.local.solutions.codednodegraph. RestrictCodedNodeGraphByAssociationExerciseSolution

69. Lesson 4: Hands-on Exercises Coded Node Graph Sandbox Sandbox Suggestions: We skipped the ResolveCodedNodeGraphListExercise, but covered it’s specifics in the GraphFocus exercise. You may go back and complete it now if you want more experience with resolving graphs. Try reversing the resolution of the coded node graph by manipulating values in the resolveToList method call. Try resolving a small list of concept codes and calling the restrictToCodes method on coded node graph

70. Lesson 4: Hands-on Exercises Example 12: LexBIG Service Manager Next navigate to the package lexevs.bootcamp.local.examples.lexbigservicemanager Get a LexBIGServiceManager instance -- open the class, GetLexBIGServiceManager. The operative statement is: return lbs.getServiceManager(null); This interface handles a number of administrative duties for the terminology service.

71. Lesson 4: Hands-on Exercises Example 12: LexBIG Service Manager Tag a coding scheme – open the class TagCodingScheme. You can tag a coding scheme as follows: LexBIGServiceManager lbsm = getLexBIGServiceManager(lbs); AbsoluteCodingSchemeVersionReference reference = Constructors.createAbsoluteCodingSchemeVersionReference( BootcampConstants.SAMPLE_ONTOLOGY_URI, BootcampConstants.SAMPLE_ONTOLOGY_VERSION); lbsm.setVersionTag(reference, "Bootcamp");

72. Lesson 4: Hands-on Exercises Self Study Exercises: LexBIG Service Manager Now navigate to lexevs.bootcamp.local.exercis es.lexbigservicemanager and do the exercises there. When you have completed all the exercises run them as a JUnit

73. Lesson 4: Hands-on Exercises LexBIG Service Manager Sandbox Sandbox: There isn’t a sandbox package for this but you can use the LexBIGService sandbox, get your own copy of the LexBIGServiceManager, and try some administrative tasks if you want. Just keep in mind, if you remove or deactivate or remove the coding scheme you will have to activate or load it again.

74. Lesson 4: Hands-on Exercises Extensions This exercise set requires: completing a set of exercises, running a build script and, executing the exercise as a java application.

75. Lesson 4: Hands-on Exercises Extensions There are two ways to register extensions in LexEVS. You can register them internally or externally. Internally registered extensions are registered by the resource manager and are considered to be an integrated part of the code base Externally registered extensions are registered using an xml file to contain extension class details.

76. Lesson 4: Hands-on Exercises Extensions We’ve discussed the LexBIGServiceConvenienceMethods as a type of generic extension Functionally it is just a layer of code over the top of LexEVS method calls that is integrated into the code base. It only looks like an extension when we call it. We don’t need to do anything special to register it – and why is that?

77. Lesson 4: Hands-on Exercises Extensions Because the Resource Manager calls a register() method on certain extension classes at startup and registers those extensions that are integrated into the code base. private static void registerExtensions() throws LBParameterException, LBException { // sort extensions EntityDescriptionSort.register(); …… // load extensions TextLoaderImpl.register(); UMLSLoaderImpl.register(); …… // Generic Extensions LexBIGServiceConvenienceMethodsImpl.register(); ……

78. Lesson 4: Hands-on Exercises Example 13: Extensions So we won’t need a special build script to build the LexBIGServiceConvenienceMethods. But we will need to instantiate like any other extension to LexEVS. Navigate to the bootcamp-extension folder and to lexevs.bootcamp.local.examples.extension. lexbigserviceconveniencemethods in your Eclipse workspace. Open the class GetLexBIGServiceConvenienceMethods. Get an instance of the LexBIGServiceConvenienceMethods LexBIGService lbs = getLocalLexBIGService(); LexBIGServiceConvenienceMethods lbscm = getGenericExtension( "LexBIGServiceConvenienceMethods", LexBIGServiceConvenienceMethods.class, lbs);

79. Lesson 4: Hands-on Exercises Example 13A: Extensions Open the class GetPathToRoot. Get the path to the root node from this concept: return lbscm.getHierarchyPathToRoot( BootcampConstants.SAMPLE_ONTOLOGY_NAME, // Sample owl null, // coding scheme version null, // Hierarchy id "Muscle", // concept code false, // concept resolution flag HierarchyPathResolveOption.ALL, // constrains no. of paths null); // association qualifiers

80. Lesson 4: Hands-on Exercises Example 13B: Extensions Open the class GetRoots. Get the root nodes of an ontology: return lbscm.getHierarchyRoots( BootcampConstants.SAMPLE_ONTOLOGY_NAME, // Sample owl null, // version null); // Hierarchy Id

81. Lesson 4: Hands-on Exercises Self Study Exercises: Extensions Now navigate to lexevs.bootcamp.local.exercis e.extension. lexbigserviceconveniencemethods and do the exercises there. When you have completed each class you can run this portion of the Exercise as a JUnit.

82. Lesson 4: Hands-on Exercises Extensions Sandbox Sandbox: There is much to play with here, but there is no Sandbox. If you want to try out some other Convenience Method navigate to the solutions package and swap methods there. We may have to limit the time we have here depending on our overall progress.

83. Lesson 4: Hands-on Exercises Custom Extensions Now we’ll create our own custom extensions This process is largely laid out for us in our examples but here is some background and an overview: LexEVS has a custom classloader that looks for types of the Extension class in the classpath. We create those extensions first by creating classes that implement the Extension class, then by creating an xml file that defines the extension. We use an ant script in this case to package the classes and the extension.xml into a jar and place them in the classpath. When the JVM starts up the LexEVS classloader loads and registers the extensions in the LexEVS runtime.

84. Lesson 4: Hands-on Exercises Example 14: Custom Extension Implement an Extension Class Navigate to lexevs.bootcamp.local.examples.extension. customextension.json.codingschemes Here we extend public interface GetCodingSchemeJsonService extends GenericExtension { public String getCodingSchemesInJson() throws Exception; Then Implement this interface public class GetCodingSchemeJsonServiceImpl implements GetCodingSchemeJsonService

85. Lesson 4: Hands-on Exercises Example 14: Custom Extension Next navigate to lexevs.bootcamp.local.examples.extension.customextension. json Here we describe the extensions we want to register like this: A sample Extension to search Google in JSON format. Mayo Clinic lexevs.bootcamp.local.examples.extension.customextension.j son.googlesearch.GoogleSearchJsonService lexevs.bootcamp.local.examples.extension.customextension.json.g ooglesearch.GoogleSearchJsonServiceImpl GenericExtension

86. Lesson 4: Hands-on Exercises Example 14-15: Custom Extension Navigate to the lexevs.bootcamp.local.examples.extension. customextension.json.codingschemes Open the file GetCodingSchemeJsonServiceImpl.java Here we have created an extension that is tied directly into the LexBIG service interface. This doesn’t have to be the case – navigate next to lexevs.bootcamp.local.examples.extension. customextension.json.googlesearch The implementations of this interface and class have no LexEVS dependencies except that they extend and implement GenericExtensions. These classes are complete and the extension.xml document is properly composed. Navigate to the lexevs.bootcamp.local.examples.extension.customextension package and run the RunCustomExtension.java application as a java application

87. Lesson 4: Hands-on Exercises Exercise 15: Custom Extension What happened? The classes and extension.xml still need to be placed in the runtime classpath of the LexEVS service instance so the custom classloader can find them and register them. Run the build script “build-examples” When it’s finished run the RunCustomExtension application again.

88. Lesson 4: Hands-on Exercises Exercise 16: Custom Extension Time to write our own extensions: Navigate to the lexevs.bootcamp.local.exercise.extension. customextension.gui package and implement the methods in the LexEvsSimpleSearchToolImpl.java class When finished, navigate to the lexevs.bootcamp.local.exercise.extension. customextension package and add content to the appropriate attributes and tags in the “extension.xml” file. Run the “build-exercise” build script. When it finishes navigate to lexevs.bootcamp.local.exercise.extension. customextension and run RunExercise as a java application.

89. Lesson 4: Hands-on Exercises Custom Extension Sandbox Sandbox Suggestions: Any “Hello World” application would be fine, but try to use the LexEVS interface in some way.

90. Lesson 5: Using the Loaders When you complete this lesson you will be able to: Use end user scripts and wrappers to load content to a LexGrid data base. Have a basic understanding of the use of preference and manifest documents for use of loading and manipulating content in LexEVS.

91. Lesson 5: Using the Loaders Loader Overview Loaders are extensions by definition. Otherwise they are integrated into the Terminology Service and as such need no special treatment. They are accessed as extensions from the LexBIGServiceManager since they are Administrative applications. So while existing loaders are integrated into the code base, and are registered at startup, plug-in loader extensions can be created and accessed via the same service manager method which employs the custom classloader under the covers to register and load any loader extensions.

92. Lesson 5: Using the Loaders Exercise 1: Loader Extensions Open a command window and navigate to C:\Program Files\LexGrid\LexEVS\5.0.0\admin Run the following command: When it completes run the command “ListSchemes.bat” and you should see the following: This slide contains screen prints of loader execution on the command line. The first screen print shows the following command line in a terminal screen LoadOBO.bat –in “C\Program Files\LexGrid\LiesEVS\5.0.0\test\resources\testData\cell.obo The second screen print shows a table in a terminal screen that contains the Thesaurus.owl and the cell terminology

93. Lesson 5: Using the Loaders Loader Requirements Loaders require at least the path to source as input All loaders accept some kind of load time modification using the manifest file. Manifest files can also be applied post load. Some loaders accept additional preferences for load time. Activation of the coding scheme is optional RRF files require an –s option to point to the coding scheme SAB in the MRSAB.RRF file.

94. Lesson 5: Using the Loaders Command Line Wrappers LexEVS provides end user wrappers and shell scripts to allow users to load content without creating their own end user API for loading terminologies. These wrappers have relatively consistent parameter options between them: -in -mf -lp -a (required for any browsing activity) -t Examples: Production, Pending, Retired, TEST

95. Lesson 5: Using the Loaders Manifests Some owl sources, like the NCI Thesaurus, require them. They allow for pre and post configuration of coding scheme metadata. This is vital when coding scheme metadata does not exist in the source but is required for a coding scheme to load. It is also helpful when coding scheme metadata does not map easily to the LexGrid model.

96. Lesson 5: Using the Loaders Example 2: Using Manifest Earlier we loaded a sample NCI owl file. Normally LexEVS will not allow the user to load a duplicate coding scheme to the terminology service. We’ll load it using a manifest to change it’s name and version in the metadata portion of the coding scheme load. Find the copy of sample-manifest.xml and copy it to the test/resources/testData folder, overwriting the existing copy. Open this xml file up in a text editor and edit the content of the codingScheme and representsVersion tag with your own content. My Nice Little NCIt 1.0.1

97. Lesson 5: Using the Loaders Exercise 2: Using Manifest If you have finished your edit, then run the owl loader pointing it at the sample.owl But add a new line pointing at the newly edited manifest This slide has screen shots of a command terminal with the lines: LoadOwl.bat –in “C:\Program Files\LexGri\LexEVS\5.0.0\test\resources\testData\sample.owl” and -mf “C:\Program Files\LexGrid\LexEvs\5.0.0\test\resource\testDatta\sample-manifest.xml Documentation of the manifest and how is used can be found at the Vocabulary Knowledge Center here: https://cabig- kc.nci.nih.gov/Vocab/KC/index.php/LexEVS_5.0_Administration_Guide#LexEVS_ Configuration_Options

98. Lesson 5: Using the Loaders Manifest Configuration Options We have other configurable options for terminology loads. For OWL and Semnet loaders we have options for configuring how concept data and relationships are loaded using an xml formatted preferences file. For example open the OwlPrefs.xml file in /test/resources/testData : currency true units max currency

99. Lesson 5: Using the Loaders Manifest Configuration Options We’ll look at a couple of these options for the OWL loader – all of which are best documented in the documentation tags of the preferences schemas here: http://informatics.mayo.edu/LexGrid/index.php?page=lexex So for currency We have: This preference allows for entity codes to be derived from a specific RDF property. The provided string is interpreted as a regular expression to be compared against properties assigned to each processed class.

100. Lesson 5: Using the Loaders Using Manifest & Preferences Some users would like to configure which nodes are designated as the roots of hierarchies – those would be set inside this tag: (C1234|C45878|C78136|C3240) The NCI Thesaurus OWL source contains embedded xml property definitions that can be enabled using this tag: true

101. Lesson 5: Using the Loaders Review 1  All loaders require that you provide a path to source, what other options are available upon loading?

102. Lesson 5: Using the Loaders Answer 1  All loaders require that you provide a path to source, what other options are available upon loading? Activate Load Preferences Load Manifest Apply a tag to the terminology

103. Lesson 5: Using the Loaders Review 2  A Loader manifest can provide configurable changes to coding scheme metadata at load time. Can you apply these changes post load as well?

104. Lesson 5: Using the Loaders Answer 2  A Loader manifest can provide configurable changes to coding scheme metadata at load time. Can you apply these changes post load as well? Yes, metadata of a coding scheme can be modified post load. However if a coding scheme is lacking metadata required for the load to complete successfully a manifest of some kind will have to be applied at load time.

105. Lesson 5: Using the Loaders Review 3  If we have a manifest to apply at load time, why is a preferences file at load time useful?

106. Lesson 5: Using the Loaders Answer 3  If we have a manifest to apply at load time, why is a preferences file at load time useful? Preferences are applied to coding scheme content and allow LexEVS to configure how elements in the source are loaded. This means some elements may be loaded differently, or not at all depending on preference settings.

107. Lesson 6: Loader Extensions in the lbGUI When you complete this lesson you will be able to: Use the GUI to load a terminology Understand how dimensions of the API work without writing code. Use set manipulations on coded node sets See how graph restrictions change the structure of a node graph

108. Lesson 6: Using the lbGUI lbGUI Overview The lbGUI (LexBIG GUI) is a developer prototype designed for the expert user. It’s easier than command line calls. Mirrors most LexEVS API calls. It can be used to do loads with options of the command line.

109. Lesson 6: Using the lbGUI Launching the lbGUI Navigate to C:\Program Files\LexGrid\LexEVS\5.0.0\gui Double click on the Windows-lbGUI.bat file: This slide shows the files in the gui directory with the Windows-lbGUI.bat file highlighted.

110. Lesson 6: Using the lbGUI Launching the lbGUI When the application opens enable the admin options in the command menu: This slide shows the upper left hand of the lbGUI Application with the Enable Admin Options menu option selected.

111. Lesson 6: Using the lbGUI Launching the lbGUI Highlight the sample you just loaded and remove it by highlighting it and clicking the remove button. This slide shows the table with the available code systems in lbGUI. To the right a column of buttons has the “Remove” button circled in red. The Thesaurus.owl code system is highlighted.

112. Lesson 6: Using the lbGUI Loading a Terminology This slide shows a pop up menu from the lbGUI with the Load OWL option selected.

113. Lesson 6: Using the lbGUI Loading a Terminology We’ll now load the sample.owl with a preferences and a manifest file; however, but we’ll use the lbGUI this time. This could have been done from the command line as well. Browse to the following files in order for loading source, preferences and manifest: sample.owl OwlPrefs.xml sample-manifest.xml Click on the load button and wait for the load to complete.

114. Lesson 6: Using the lbGUI Loading a Terminology We should now have three terminologies loaded. We’ll use the GUI to browse these terminologies and otherwise manipulate them. In particular everyone should have the cell.obo file loaded. If not, let’s load it now. If that is done, then activate it by highlighting the cell ontology in the GUI table and pressing the activate button in the right hand column of buttons. This slide contains a view of the lbGUI table with three terminologies, cell, and two verions of the Thesaurus.owl loaded.

115. Lesson 6: Using the lbGUI Using the GUI lbGUI – Use it to help formulate method calls in your code. Remember the restrict and resolve pattern we have been discussing for optimizing queries. We’ll use the GUI to demonstrate this pattern as well as other patterns of use in the LexEVS API.

116. Lesson 6: Using the lbGUI Using the GUI First Highlight the cell ontology in the table of the GUI. This slide shows the Available Coding Schemes table with the cell terminology highlighted.

117. Lesson 6: Using the lbGUI Using the GUI Then press the “Get Code Set” button on the right hand of the GUI. This slide shows the left hand column of buttons with Get Code Set circled in red.

118. Lesson 6: Using the lbGUI Using the GUI A coded node set designation should appear in the lower left hand side of your GUI. Click on this to highlight it. This slide shows the Selected CodedNodeSets and CodedNodeGraphs window from the lower left hand side of the GUI with the cell node set selected.

119. Lesson 6: Using the lbGUI Using the GUI On the lower right hand side of the GUI press the “Add” button. A pop up wizard will appear that looks like this. It opens to a default restriction to matching designations

120. Lesson 6: Using the lbGUI Using the GUI Matching to designations will effectively restrict the text search to presentation type properties. We will be using the standard Lucene query which works as a kind of “contains” search but can be modified by adding special characters and logic designations. We’ll leave the language field blank which effectively sets the search to this coding schemes default language. The designation type gives an option to search for preferred, non- preferred or both. Enter the word “bone” into the Match Text field and press the “OK” button on the bottom right side of the pop up box. These fields represent parameters in a coded node set method call restricting the coded node set to those values with designations matching the text criteria entered.

121. Lesson 6: Using the lbGUI Using the GUI This restriction now appears in a window as the beginning of a list of restrictions you might want to apply to this coded node set. This slide shows the Restrictions window from the lower right hand side of the gui with one restriction in the list “Designations Matching ‘bone’ On the side are three buttons: “add’, “edit”, “remove”. Underneath are two more buttons, “Set Sort Options” and “Resolve Code Set.”

122. Lesson 6: Using the lbGUI Using the GUI Resolve the coded node set by clicking on the resolve button. In the resulting popup window, click on the first concept in the left hand column This slide shows the Result Browser with two results showing in the left hand column.

123. Lesson 6: Using the lbGUI Using the GUI Notice that, even though we are resolving a set we still get results for the directly related neighbors of this concept Notice also that we are getting the fully resolved entity for this concept This slide shows the Result Browser with two results showing in the left hand column.

124. Lesson 6: Using the lbGUI Using the GUI Now highlight the NCI Thesaurus scheme (the first one we loaded without manifest or preference files), get a coded node set, set the restriction in exactly the same way, and resolve the set. This slide shows how the same restriction would be resolved in the NCI Thesaurus. The returned concepts and displayed graph have different values.

125. Lesson 6: Using the lbGUI Using the GUI We now have two coded node sets from two different terminologies resolved to the same restrictions. We might make them more useful to an end user by placing them within the context of a single coded node set. We can do that by using control-left-click to highlight the two coded node sets and then pressing the union button. The resulting union appears as a third coded node set in the list. We could also get a node graph from the owl ontology and use the node set from that ontology as source and/or target codes for the graph by highlighting set and graph and clicking on one of the restrict to codes buttons that appears. The result is a new coded node graph in the list of graphs and sets.

126. Lesson 6: Using the lbGUI Resolving a Coded Node Graph If you haven’t already got a coded node graph from the cell ontology – then get one now. Highlight the graph in the list of sets and graphs. Resolve the graph by clicking on the “Resolve as Graph” button in the lower right hand side of the GUI. This will nominally give you a coded node graph of the entire terminology. In fact it gives you a maximum of 500 graph nodes with their associations and essentially an unresolved concept for each, which you can click on to get resolved completely.

127. Lesson 6: Using the lbGUI Resolving Restrictions The restrictions window in the lower left hand side provides the user with parameter entry fields for what is essentially a resolve to list method call. You can choose or enter a Relations Container, a Focus Code, a Focus Code System, a Maximum Resolve Depth, an option to resolve the graph towards source and/or target concepts, and a maximum to return. This slide shows the resolution fields for a coded node graph. It has text fields for Relations Container, Focus Code, Focus Code System, Max Resolve Depth and Max to Return. It has a check box for each of the values Resolve Forward and Resolve Backwards. It contains three buttons on the lower right Set Sort Options, Resolve as Set and Resolve as Graph.

128. Lesson 6: Using the lbGUI Resolving Restrictions If you click on the add button next to the restrictions list window you get options for restricting a coded node graph. Click cancel and return to the previous window. This slide depicts a popup wizard for coded node graph restrictions. The scroll bar is marked Restriction Type and is set to Restrict to associations. Below in the Associations field are three associations: -multi-assn-@-root-, develops_from, and is_a. It has an ok and a cancel button.

129. Lesson 6: Using the lbGUI Resolving Restrictions At this window type in a Focus Code value of CL:0000049 Then change the value of the Max Resolve Depth from -1 to 1. Check both “Resolve Forward” and “Resolve Backward” check boxes Press the button “Resolve as Graph” This slide brings us back to the coded node graph resolution window which has the text fields set for focus code at CL:0000049 and the focus code system set to cell. The max resolve depth is set to 1 and the Max to return is set to 500. Both Resolve forward and resolve backward boxes are checked. The Resolve as Graph button is circled in red.

130. Lesson 6: Using the lbGUI Graph Display Notice the two kinds of associations displayed in the result browser. Close this result browser and click on the “Add” button in the restrictions window.

131. Lesson 6: Using the lbGUI Graph Display: Associations Restrictions Select the association “develops_from” in the associations list. Click “Ok” to select this as the association to restrict to.

132. Lesson 6: Using the lbGUI Graph Display: Associations Restrictions This results in a node graph restricted to develops_from associations only.

133. Lesson 6: Using the lbGUI Review In LexEVS restrictions and their parameters define the result set in a way that has particular meaning to the user. For instance: The user wants a result set that has “myocardial infarction” in the presentation, not “fungal infection.” The results are tailored to a specific request for defined information that provides meaningful results to the user. In LexEVS resolving results largely defines the structure of the results by the metrics of list size and object volume. Resolving lists provides utility to the developer, protecting performance and increasing usability. The resulting list of objects or iterator for objects allows the developer to show the user results that have minimum identification criteria shown in a small enough quantity to allow good performance -- insuring the user can identify and select from results that can be resolved fully and quickly.

134. Lesson 6: Using the lbGUI Further Study Options Not included but possible further study options for you are: Directed Acyclic Graph walking. Lazy loading of Node Graphs Iterator management Creating your own loader Extending the new Spring Batch Loader Framework (this is a LexEVS 5.1 implementation)

135. 135 LexEVS in a Local Environment Questions?