1. CaDSR Java API Tutorial by John Kramer Tutorial Document by Manny Roble with assistance from Gene Levinson, and John Kramer

2. Some Project Background The project for which we used the caDSR Java API was a CDE search tool similar to the NIH CDE Browser, but with functions which were customized to meet the needs of our forms development team. The following were some of the requirements for this project: Allow the user to search for Data Elements by document text, long name, or preferred definition. If search text is contained in any of these fields, the data element should be returned. Only search for data elements in the CTEP context. Perform searches with AND and OR operators.

3. Project Background cont. Search only for data elements with specified workflow statuses. Link returned data elements to object in our local Oracle database. Allow the user to see both local data element and CDE from the NIH side by side. Allow the user to see permissible values for data elements and allow them to search for data elements that contain specified permissible values. Search for data elements which have been updated since a specified date.

4. Introduction to the tutorial For these tasks, we used the NIH caBIO Java API. We used the API primarily to query the caDSR repository for Common Data Elements. The information in this presentation and in the tutorial are geared toward retrieving data from the caDSR repository (particularly Common Data Elements), though the information is also more generally useful as an introduction to using the caBIO API.

5. Getting Started with the API The first thing that you need is the caBIO.jar file which contains the necessary classes. The jar file can be downloaded from the NCICB web site. Most of the common classes needed should be made available by the following import statements: import gov.nih.nci.caDSR.bean.*; import gov.nih.nci.caDSR.search.*; import gov.nih.nci.common.search.*; You will also have to modify your java.policy file to allow for connection to the RMI server. There is a sample policy file excerpt in the CDE Tutorial document. There is reference class documentation for the API at http://ncicb.nci.nih.gov/content/coreftp/caBIO2-1_JavaDocs.

6. The Domain Objects Objects in the caDSR repository are direct or indirect subclasses of the DomainObject abstract class. DataElement, ReferenceDocument, and PermissibleValue are all examples of Domain Objects in the caDSR repository. Searches to the NIH database using the caBIO Java API will return a collection of the DomainObject that you are searching for.

7. The getXXX methods Each DomainObject has a series of getter methods to retrieve the values of the attributes for that object. For instance, the DataElement object has methods getLongName() and getPublicId() which will retrieve the longName and the publicId for the DataElement respectively. There are also methods to retrieve related objects for a given Domain Object. For example, there is a method getContext() on the DataElement object that will return the related Context object.

8. The getXXX methods cont. In some cases there is a 1 to many relationship between the primary object and a related object. In these cases, the method to retrieve the related objects will return an array of such objects. The getReferenceDocuments() method is an example of such a method on the DataElement object. One thing to note: These methods that retrieve related objects will dynamically query for the objects when the method is invoked. This can take a little bit of time. This can be very significant when processing groups of these requests. For instance, if you have an array of 50 data elements, and want to retrieve the Context for all 50 (to print out the context name, for instance), this may take a little while.

9. The SearchCriteria Object Each domain object in the caDSR repository has a corresponding SearchCriteria object The SearchCriteria object encompasses the logic used to search the NIH database You can tell the SearchCriteria object exactly what criteria you want to use to select domain objects from the NIH database and when you perform your search In most (maybe all) instances, the name of the SearchCriteria is the name of the corresponding domain object with SearchCriteria added onto the end. For instance, the search criteria object for DataElement is DataElementSearchCriteria.

10. Searching and Search Results To perform a search, you can use the search() method on your domain object. This method takes as a parameter the SearchCriteria object which you wish to use in your search. When you perform your query, your results will be returned in a SearchResult object. There is a method getResultSet() on the SearchResult object which will then return an array of domain Objects that are contained in your result set. By default, a maximum of 1000 results will be returned. There is a method setMaxRecordset() on the SearchCriteria object which you can use to change this number.

11. Showing Search Results // Go through Results StringBuffer stats = new StringBuffer(); if (result != null) { DataElement[] dElements = (DataElement[])result.getResultSet(); for (int i = 0; i < dElements.length; i++) { DataElement crf = dElements[i]; stats.append("\n------"); stats.append("\n").append("wfStatusName: ").append(crf.getWorkflowStatusName()); stats.append(" | ").append("CDE ID: ").append(crf.getPublicId()); Context ctx = crf.getContext(); stats.append(" | ").append("Context: ").append(ctx.getName());

12. Showing Search Results cont. stats.append("\n").append("Pref Def: ").append(crf.getPreferredDefinition()); stats.append("\n").append("LongName: ").append(crf.getLongName()); } // End of For stats.append("\n").append(" DataElement Results: ").append(dElements.length); } // End of (result != null) System.out.println("* * * SearchResults: " + stats.toString()); }

13. The setXXX methods Each SearchCriteria object has a number methods which allow you to directly set attributes to search on. The DataElementSearchCriteria object, for instance, has methods such as setLongName(), setPreferredDefinition(), and setPublicId(). If you use any of these methods, only Data Elements which match the attributes that you selected will be returned when you perform your search. For instance, if you invoke setLongName(“*adverse event*”), only DataElements where the longName matches *adverse events* will be selected. Note that the * is treated as a wild card character in searches.

14. A Very Simple Example (This is from example 2 of the CDE tutorial) This example will search for all Data Elements which have a workflow status of “RELEASED_NON_CMPLNT”. DataElement de = new DataElement(); DataElementSearchCriteria deCrit = new DataElementSearchCriteria(); deCrit.setWorkflowStatusName( "RELEASED_NON_CMPLNT"); // Do Search showResults(de.search(deCrit));

15. Here is an only slightly more complex example (This is from example 4 of the CDE tutorial document) This example will search for data elements whose long name AND preferred definition are LIKE *leuk*. DataElement de = new DataElement(); DataElementSearchCriteria deCrit = new DataElementSearchCriteria(); String sTxt1 = "*leuk*"; deCrit.setLongName(sTxt1); deCrit.setPreferredDefinition(sTxt1); // Do Search showResults(de.search(deCrit));

16. Searching Related Domains To search a related domain object, you must first instantiate a SearchCriteria element for the related domain object and that add it to your DataElementSearchCriteria. For instance, to search the Context domain object, first instantiate a ContextSearchCriteria class and then add this SearchCriteria object to your DataElementSearchCriteria object.

17. A Simple Example of searching a related domain object (This is from example 1 of the CDE tutorial) This will search for all data elements which exist in the Context CTEP. DataElement de = new DataElement(); DataElementSearchCriteria deCrit = new DataElementSearchCriteria(); // Adding Context SearchCriteria and set to "CTEP" ContextSearchCriteria ctxCrit = new ContextSearchCriteria(); ctxCrit.setName("CTEP"); // increase max number of returns deCrit.setMaxRecordset( new Integer(10000)); deCrit.putCriteria("context",ctxCrit); // Do Search showResults(de.search(deCrit));

18. A Note on the above examples The setXXX methods are very convenient and easy to use but are limiting in what they allow us to do. For one thing, if you invoke two or more setXXX methods, the default operator for selecting objects will be AND. Using these methods would not allow us to search for data elements where longName OR preferredDefinition is like “*leuk*”, for instance. To perform more complex searches, we can use the AttributeCriterion(), AssociationCriterion(), and/or CriteriaGroup() objects which will be explained next.

19. The AttributeCriterion object Another way of specifying searches on domain object attributes is to use the AttributeCriterion object. This is more verbose than using the setXXX methods, but also allows for more flexibility when constructing searches. An AttributeCriterion can be added as a Criterion to the SearchCriteria object with the putCriterion() method.

20. AttributeCriterion cont. To create an AttributeCriterion you will need to specify three things: the name of the attribute on the domain object, the comparison operator (these operators are constants in the Criterion class), and a collection of values to select when searching. For example, the following statements will create an AttributeCriterion object which will select objects where longName is LIKE “*leuk*”: Collection patternCollection = new ArrayList(); String pattern = “*leuk*”; patternCollection.add(pattern); AttributeCriterion att1 = new AttributeCriterion(“longName”, Criterion.LIKE,patternCollection);

21. A Simple AttributeCriterion example (This example is not in the CDE tutorial) This will select all Data Elements where longName is like *radiation* DataElement de = new DataElement(); DataElementSearchCriteria deCrit = new DataElementSearchCriteria(); String sTxt1 = "*radiation*"; Collection sStr = new ArrayList(); sStr.add(sTxt1); AttributeCriterion att1 = new AttributeCriterion("longName", Criterion.LIKE, sStr); deCrit.putCriterion(att1); showResults (deCrit.search());

22. The AssociationCriterion An AssociationCriterion is a criterion for an associated domain object to the object that you are searching. You can use this for search related domain objects. An AssociationCriterion wraps a SearchCriteria object. You can create an AssociationCriterion out of a SearchCriteria very simply. The following example code will create an AssociationCriterion and add the ContextSearchCriteria object (ctxCrit) to it: AssociationCriterion crit = new AssociationCriterion(“context”, ctxCrit);

23. CriteriaGroup The CriteriaGroup object can be used to formulate more complex queries involving AND and OR operators. Each CriteriaGroup has an operator associated with it which is either Criterion.AND or Criterion.OR. You can then add any number of AttributeCriterion, AssociationCriterion, or CriteriaGroup objects to the CriteriaGroup. This also allows for the possibility of complex searches using nested Criteria Groups.

24. A CriteriaGroup example (This is from example 5 of the CDE tutorial) This will search for data elements where the longName and preferredDefinition are LIKE “*leuk*”. The results of this search are the same as in example 4. DataElement de = new DataElement(); DataElementSearchCriteria deCrit = new DataElementSearchCriteria(); String sTxt1 = "*leuk*"; Collection sStr = new ArrayList(); sStr.add(sTxt1); AttributeCriterion att1 = new AttributeCriterion("longName", Criterion.LIKE, sStr);

25. CriteriaGroup example continued… AttributeCriterion att2 = new AttributeCriterion("preferredDefinition", Criterion.LIKE, sStr); Collection cList = new ArrayList(); cList.add(att1); cList.add(att2); CriteriaGroup cGroup = new CriteriaGroup(Criterion.AND, cList); deCrit.addCriteriaGroup(cGroup); // Do Search showResults(de.search(deCrit));

26. A Listing of Criterion operators Comparison Operators Criterion.EQUAL_TOCriterion.NOT_EQUAL_TO Criterion.GREATER_THAN Criterion.GREATER_THAN_OR_EQUAL_TO Criterion.LESS_THAN Criterion.LESS_THAN_OR_EQUAL_TO Criterion.BETWEENCriterion.NOT_BETWEEN Criterion.LIKECriterion.NOT_LIKE Conjunctive Operators Criterion.AND Criterion.OR

27. A Sample Problem (Example 9) Lets suppose we wanted to develop a query that would search for data elements that contained the text string “*leuk*” in their long name or in their document text. One thing to note is that the Document Text is not stored in the DataElement object but in the related ReferenceDocument object. The equivalent pseudo sql code would be : SELECT * FROM DataElements, ReferenceDocument WHERE (DataElement.longName LIKE "%leuk%" OR (ReferenceDocument.docText LIKE "%leuk%" AND ReferenceDocument.type = "LONG_NAME")) AND DataElement JOIN ReferenceDocument

28. Problem cont. We are going to need two Criterion objects: an AssociationCriterion which encapsulates a ReferenceDocumentSearchCriteria for searching the document text and an AttributeCriterion for searching on longName. We can then add these two Criterion objects to a CriteriaGroup with the Criterion.OR operator. The CriteriaGroup object can then be added to our DataElementSearchCriteria and then we are ready to perform our search

29. Problem cont.

30. Code for example 9 (from tutorial document) try { DataElement de = new DataElement(); DataElementSearchCriteria deCrit = new DataElementSearchCriteria(); String sTxt1 = "*leuk*"; Collection sStr = new ArrayList(); sStr.add(sTxt1); // Set longName attribute which belongs to DataElement domain object AttributeCriterion att1 = new AttributeCriterion("longName", Criterion.LIKE, sStr);

31. code continued // Create search criteria for ReferenceDocument.document text (related //domain object) You need to use AssociationCriterion to do this ReferenceDocumentSearchCriteria docCrit = new ReferenceDocumentSearchCriteria(); docCrit.setType("LONG_NAME"); docCrit.putCriterion("doctext",Criterion.EQUAL_TO, sTxt1); AssociationCriterion att2 = new AssociationCriterion(); att2.setSearchCriteria(docCrit);

32. code continued again // Create OR CriteriaGroup add the two attributes you are using // in your search to the DataElementSearchCriteria object Collection cList = new ArrayList(); cList.add(att1); cList.add(att2); CriteriaGroup cGroup = new CriteriaGroup(Criterion.OR, cList); deCrit.addCriteriaGroup(cGroup); // Do Search showResults(de.search(deCrit)); } catch (Exception e) { e.printStackTrace(); }

33. Another Problem (Example 10) Now lets suppose that we further want to restrict our search to those Data Elements where context=‘CTEP’. So the pseudo sql code for our problem now looks like: SELECT * FROM DataElements, ReferenceDocument, Context WHERE (DataElement.longName LIKE "%leuk%" OR DataElement.preferredDefinition LIKE "%leuk%" OR (ReferenceDocument.docText LIKE "%leuk%" AND ReferenceDocument.type = "LONG_NAME")) AND Context.name=‘CTEP’ AND DataElement JOIN ReferenceDocument AND DataElement JOIN Context

34. Another Problem cont.

35. Code for example 10 (from tutorial document) try { DataElement de = new DataElement(); DataElementSearchCriteria deCrit = new DataElementSearchCriteria(); String pattern = "*leuk*"; Collection orGroup = new ArrayList(); orGroup.add( new AttributeCriterion( "longName", Criterion.LIKE, Arrays.asList(new String[] { pattern }))); orGroup.add( new AttributeCriterion("preferredDefinition", Criterion.LIKE, Arrays.asList(new String[] { pattern})));

36. Example 10 code cont. ReferenceDocumentSearchCriteria rdsc = new ReferenceDocumentSearchCriteria(); rdsc.setDoctext(pattern); rdsc.setType("LONG_NAME"); orGroup.add(new AssociationCriterion("referenceDocuments", rdsc)); ContextSearchCriteria csc = new ContextSearchCriteria(); csc.setName("CTEP"); // increase max number of returns deCrit.setMaxRecordset(new Integer(10000));

37. Example 10 code cont. Collection andGroup = new ArrayList(); andGroup.add(new AssociationCriterion("context", csc)); andGroup.add(new CriteriaGroup(Criterion.OR, orGroup)); DataElementSearchCriteria desc = new DataElementSearchCriteria(); desc.addCriteriaGroup(new CriteriaGroup(Criterion.AND, andGroup)); // Do Search showResults(de.search(deCrit)); } catch (Exception e) { e.printStackTrace(); }}

38. One Last Thing: Association Chaining So far we have seen how to limit our searches for DataElement objects based on values for related ReferenceDocument and Context objects. This process can be extended to other directly related domain objects. Sometimes you may want to limit your search based on the value for a domain object which is not directly related to the domain object but is related through one or more intermediary objects. For instance, in our program, we wanted to be able to select only DataElements which had certain PermissibleValues.

39. Association chaining cont. The PermissibleValue domain object is related to the DataElement object through the EnumeratedValueDomain and ValueDomainPermissibleValue objects. What we have to do is form a “chain” of SearchCriteria objects linking the PermissibleValueSearchCriteria to the DataElementSearchCriteria. The same thing can also be accomplished with a “chain” of AssociationCriterion elements.

40. Association chaining cont. The following code will construct a DataElementSearchCriteria to perform this task: DataElementSearchCriteria deCrit = new DataElementSearchCriteria(); String pattern = "*radiation*"; PermissibleValueSearchCriteria pvCrit = new PermissibleValueSearchCriteria(); ValueDomainPermissibleValueSearchCriteria vdpvCrit = new ValueDomainPermissibleValueSearchCriteria(); EnumeratedValueDomainSearchCriteria evdCrit = new EnumeratedValueDomainSearchCriteria();

41. Association Chaining cont. pvCrit.setValue(pattern); vdpvCrit.putCriteria("permissibleValue",pvCrit); evdCrit.putCriteria("valueDomainPermissibleValues", vdpvCrit); deCrit.putCriteria("enumeratedValueDomain",evdCrit);