1. LabKey Server 11.1 What’s New for Developers Josh Eckels March 17, 2011

2. 11.1 Feature List Improved look and feel GPAT data persistence in hard tables Server-side validation script enhancements JavaScript reports AJAX report designer Copy-to-study by reference Study data visualization File management UI enhancements Easier initial data import for GPAT Additional LabKey SQL syntax Agilent feature extraction (microarray) pipeline update Sequest (proteomics) pipeline update Misc assay enhancements Improved look and feel GPAT data persistence in hard tables Server-side validation script enhancements JavaScript reports AJAX report designer Copy-to-study by reference Study data visualization File management UI enhancements Easier initial data import for GPAT Additional LabKey SQL syntax Agilent feature extraction (microarray) pipeline update Sequest (proteomics) pipeline update Misc assay enhancements

3. Improved Look and Feel Now deployed on atlas-test and atlas-staging Nick presented at January Atlas developers meeting Goals – Cleaner and less distracting – More consistent – Easier for developers Continuing, multi-release effort

6. GPAT Data Persistence Similar to work done for datasets in version 10.3 Before 11.1, three forms of persistence – Metadata stored in OntologyManager – Data stored in OntologyManager (aka blender) – Data also stored in materialized temp tables In 11.1, two forms of persistence – Metadata stored in OntologyManager – Data stored only in OntologyManager maintained hard tables 100% UI and API compatibility API access is still strongly recommended for data extraction

7. Copy-To-Study by Reference Copy-to-study is mechanism for including assay data in studies Prior to 11.1, copied data rows were snapshotted into dataset As of 11.1, assay data rows are referenced by dataset as a lookup Eliminates duplicated data 100% UI and API compatibility

8. LabKey SQL – More Functions Pass-through support for more database functions asciioctet_lengthsubstr btrimoverlapsto_ascii char_lengthquote_identto_hex character_lengthquote_literaltranslate chrregexp_replaceto_char decoderepeatto_date encodereplaceto_timestamp initcaprpadto_number lpadsplit_part md5strpos

9. LabKey SQL – General Syntax Comma join syntax SELECT Col1, Col2 FROM TableA, TableB WHERE TableA.Col3 = TableB.Col4 Auto-generated expression aliases SELECT LCASE(FirstName), Age FROM People Expression1Age billy12 sally11

10. LabKey SQL – User/Group Functions USERID() – Returns id of the current user – Useful for filtering within query ISMEMBEROF(groupId) – Returns true if current user is a member of the specified group id – core.groups table contains group names and ids

11. LabKey SQL – General Syntax ISEQUAL(a, b) – Treats NULLs as equal – Equivalent to: (a=b OR (a IS NULL AND b IS NULL)) Nested joins now supported... FROM TableA LEFT JOIN (TableB INNER JOIN TableC ON...)

12. LabKey SQL – General Syntax AS is now optional for expression aliases SELECT LCASE(FirstName) AS Name FROM PEOPLE SELECT LCASE(FirstName) Name FROM PEOPLE ORDER BY clause can refer to column alias SELECT LCASE(FirstName) Name FROM PEOPLE ORDER BY Name

13. LabKey SQL - PIVOT Similar to Excel pivot tables Reshape rectangular data Related to aggregates and GROUP BY PTIDVisitIDAnalyteValue 54359645102IL-1b105 54359645102IL-2231 54359645102IL-7178 43209543314IL-2314 43209543314IL-790 43209543314IL-12p70143 PTIDVisitIDIL-1bIL-2IL-7IL-12p70 54359645102105231178 43209543314 90143

14. LabKey SQL - PIVOT SELECT PTID, VisitID, Analyte, Value FROM AssayData PTIDVisitIDAnalyteValue 54359645102IL-1b105 54359645102IL-2231 54359645102IL-7178 43209543314IL-2317 43209543314IL-790 43209543314IL-12p70143

15. LabKey SQL - PIVOT SELECT PTID, VisitID, MAX(Value) AS MyValue FROM AssayData GROUP BY PTID, VisitID PTIDVisitIDMyValue 54359645102231 43209543314317

16. LabKey SQL - PIVOT SELECT PTID, VisitID, MAX(Value) AS MyValue FROM AssayData GROUP BY PTID, VisitID PIVOT MyValue BY Analyte PTIDVisitIDIL-1bIL-2IL-7IL-12p70 54359645102105231178 4320954331431790143

17. LabKey SQL - PIVOT SELECT PTID, VisitID, MAX(Value) AS MyValue FROM AssayData GROUP BY PTID, VisitID PIVOT MyValue BY Analyte IN (‘IL-2’, ‘IL-7’) PTIDVisitIDIL-2IL-7 54359645102231178 4320954331431790

18. LabKey SQL – Parameterized Queries Pass data to custom queries Allows filtering on pre-aggregated values Declared at top of query PARAMETERS(MinValue INTEGER) SELECT PTID, VisitID, AVG(Value) AS AvgValue FROM AssayData WHERE Value > MinValue GROUP BY PTID, VisitID

19. LabKey SQL – Parameterized Query Example MinValue = 0MinValue = 125 PTIDVisitIDAvgValue 54359645102171 43209543314183 PTIDVisitIDAvgValue 54359645102204 43209543314230

20. LabKey SQL – Parameterized Values Provide as config property to JavaScript APIs Or provide as URL (GET) parameters – query.param.MinValue=100 Or use minimal UI for setting values – Intended for query authors, not end users – Parameterized queries usually require custom UI

21. JavaScript Validation and Transformation Originally added in version 10.2 Part of file-based modules Executed on server, as the user who submitted the request Row-level callbacks for insert, update, delete Associated with a specific schema/query May transform data, or simply accept/reject Supported for many but not all data types (lists, datasets, external modules, etc)

22. Validation Improvements - APIs require(“dir/dependencyName") – Include other file-based module script files Supports a subset of the JavaScript client APIs – LABKEY.ActionURL – LABKEY.Filter – LABKEY.Message – LABKEY.Query – LABKEY.Security – LABKEY.Utils

23. Validation Scripts – API Callbacks Calls that are asynchronous on client are synchronous on server Return value from client API function is its result Your callback function, if provided, is called before function returns

24. Example Validation Script var console = require("console"); var LABKEY = require("labkey"); /* Don’t allow user to delete last row */ function beforeDelete(row, errors) { // Get all rows currently in the same table var rs = LABKEY.Query.selectRows({ schemaName: 'vehicle', queryName: 'Manufacturers', columns: ['RowId','Name']}); if (rs.rows.length == 1) { // Error is not tied to a field, so use null // instead of a field name errors[null] = “Last row – can’t delete”; }

25. JavaScript Reports Analogous to R reports Alternate view of data grids Passed context object with current schema, query, filters, etc Available through Views menu – Views->Create->JavaScript View – View->My JavaScript View Name Uses revamped report builder – AJAX between source, view, and data tabs – Enhanced editor for source code

29. More Info 11.1 features have specs on our Project List – https://www.labkey.org/project/home/Developer /Projects/begin.view https://www.labkey.org/project/home/Developer /Projects/begin.view Developer documentation at www.labkey.orgwww.labkey.org – LabKey SQL reference – JavaScript and other client APIs – Some features may lag behind the release date Email me – jeckels@labkey.com

30. Questions