caDSR Freestyle Search June 11, 2009

caDSR Freestyle Search Overview Architecture Implementation Dependencies Futures 2

caDSR Freestyle Search - Overview Provides a “Google” like search across caDSR Case Insensitive Results limited to only highest ranked matches – does *not* normally return all matches Match weight a result of term sequence, intervening terms, number of occurrences, Workflow Status, Registration Status, Administered Item type, etc Results sorted descending by weight, i.e. heaviest match appears at the top of the list Does not require user to know caDSR structure or objects/attributes to perform searches 3

caDSR Freestyle Search - Overview Stakeholders: –Form Designers –Modelers –Developers –Analysts –Clinicians –Statisticians –Researchers –Curators –caBIG –NCI 4

caDSR Freestyle Search - Architecture Technologies –Java 1.5 –Javascript –HTML 4 –JDBC –Struts –EVS 4.2 5

caDSR Freestyle Search - Architecture Struts / JSP / HTML View JBoss Controller Java 1.5 Application Class, Interface Model JDBC, PL/SQL, ANSI SQL Database Oracle 10g Persist 6

caDSR Freestyle Search - Architecture Auto-deploy –Deployable via Anthill –Ant –DPROP.FILE=… build-all deploy SCM –CVS –.cvsignore for all transient files –One file, no duplicates, e.g. template.web.xml vs. web.xml All files placed in deployment-artifacts Production deployment artifacts –Accessible via links in from Anthill –Files hosted on GForge for distribution –URL references to GForge hosting for Wiki, Download, etc 7

caDSR Freestyle Search - Architecture Jboss/freestyle.war –Web Browser UI –Passes input to JAR and formats result in HTML, pure UI layer Gforge/freestylesearch.jar –API interface for searches, options, etc Bin/autorun.sh –Deploys to /local/content/freestyle/bin/. –Automated job to update search indices –Scheduled and launched by CRON every morning at 3:00 am and every hour between 8:00 am and 5:00 pm 8

caDSR Freestyle Search - Architecture tool name FREESTYLE SENTINEL … property URL … value m … … 9 Tool Options Table Tool options table hosts configuration values beyond 3 rd party requirements, e.g. XML Dynamic –Values are read as needed – user sees changes in real time –Values cached when new session created – user must close window –Values never cached with application – requires restart of JBoss

caDSR Freestyle Search - Architecture SQL script updates/sets tool option values –Updates limited to FREESTYLE tool name SQL may check database schema during deployment –E.g. When a new column is added to a table/view a SELECT using the column name will throw an error if the database is not updated before deploying the tool SQL may *never* alter schema SQL may perform data migration Must be coordinated and negotiated with caDSR database deployment scripts Index updates write current timestamp on completion 10

caDSR Freestyle Search - Implementation Project Structure –Conf Configuration files, e.g. XML, which require value substitution during build and deployment –Db-sql Scripts to correct errors in index tables –Doc Patterned after phases in development lifecycle with the addition of “Administration” for all documentation specific to NCI policies and processes and not directly pertinent to the product features Administration Construction Elaboration Inception Transition –Lib JAR files needed for building the project *but* not included in the deployment, e.g. ojdbc14.jar is deployment on Jboss and not packaged in project WAR but must be present to compile and build the WAR, allows for the separation of the build machine and the deployment target machine 11

caDSR Freestyle Search - Implementation 12 Project Structure –Scripts Console scripts to update index tables –Src Java source, more details follow –WebRoot The deployed freestyle.war content Css Html Images Js Jsp Meta-inf Web-inf –Lib –Tld

caDSR Freestyle Search - Implementation Packages –gov.nih.nci.cadsr.freestylesearch.test Automated tests –gov.nih.nci.cadsr. freestylesearch.tool Main business logic –gov.nih.nci.cadsr. freestylesearch.ui Web Browser UI using Struts –gov.nih.nci.cadsr. freestylesearch.utl Utility features, e.g. Search, results object types, etc Search entry utl/Search.java Index table –Update entry point utl/Seed.java –Configuration cont/template.seed.xml 13

caDSR Freestyle Search - Implementation Logging –freestylesearch_log.txt Jboss messages from gov.nih.nci.cadsr.freestylesearch.* –Server.log Jboss messages from 3 rd party packages, e.g. struts –Seed_log.txt Messages from the update to the index tables 14

caDSR Freestyle Search - Dependencies caDSR API –The search results are returned in Freestyle defined class objects or in AdministeredItem derived class objects per the search method used. Oracle 10g –The weight algorithm relies on calculations performed in SQL, this is necessary to avoid sending large amounts of data to the web server for weight calculations. 15

caDSR Freestyle Search - Futures Upgrade the caDSR API as needed Research use of Lucene Add “sounds like” matching Add singular/plural matching Add wildcard support Add Concept matching Add selection of indirect Admin Item type, e.g. return all DE where DEC is … Improve performance (possibly define database indexes on index table columns) Add weight calculation customizations, e.g. matches in long_name should be 2x all other columns 16