1. iPlant Tool Integration Process

2. 2 Virtual Iron 2 Big Iron Commod ity Iron Algorith ms: Big Small Data: Big Small Plant Biologists Community Computational Biologists iPlant Cyberinfrastructure High-Level View of iPlant CI

3. What we need How to download the tool or source code, including installation and usage instructions. The required version of the program. A sample dataset and the expected results to be output. The set of parameters and command line switches that match the expected execution of the tool. A brief text description of the tool, written for someone who has not used the tool before. This should include links to literature for further reading.

5. 5 A few things to keep in mind iPlant’s resources are limited iPlant cannot generate new data iPlant needs collaborators with expertise Complex data integration and interactive visualization are big problems 5 Biology and Computer Science have technical jargon: Clear communication and patience are essential!

6. JSON

9. { "name": "iPlant Collaborative", "address": { "street": "E Helen St", "number": 1657, "city": "Tucson", "state": "AZ", "ZIP": 85721 }, "contacts": [ { "first name": "Naim", "last name": "Matasci", "function": "Engagement Team Analyst", "telephone": { "prefix": 520, "number": "626-3756" }, "email": [ "nmatasci@iplantcollborative.org", "matasci@email.arizona.edu" ] }, { "first name": "Sonya", "last name": "Lowry", "function": "Director of Cyberinfrastructure Development ", "telephone": { "prefix": "520", "number": "626-9104" }, "email": "sonya@iplantcollaborative.org" } ] }

10. Tool metadata What is being integrated?

11. What? Name: Tool name Description: What the tool does Type: "executable" (more will come) [location]: Where the executable reside (we'll tell you that) Attribution: Who wrote it Version: Which version (we want to get the right thing)

12. > command [options] infiles outfiles { "name": "cat", "description": "Reads the contents of a file and writes it to standard output", "type": "executable", "location": "/bin/cat", "attribution": "A really smart UNIX guru like Rob Pike wrote the original version. A cat utility appeared in Version 1 AT&T UNIX. Dennis Ritchie designed and wrote the first man page.", "version": "7.4.1", } > cat –d infile >outfile

13. Tool Interface Metadata How is it integrated?

14. { "id":"", "name": "", "type": "", "input":[... ], "groups": [... ], "output":[... ] } Info Infiles Options > command [options] infiles outfiles Outfiles

15. { "id":"", "name": "Linux cat", "type": "cat", "input":[…], "groups":[…] "output":[…] } >cat –b filename.txt >out.txt From the metadata description

16. { … "input":[ { "id":"catin", "name":"File to read:", "type":"File", "multiplicity":"single", "order":2 }, ], "groups": […], "output":[…] } >cat –b filename.txt >out.txt

17. { … "groups": [ { "name": "Options", "id": "", "type": "step", "properties": [ { "id":"bflag", "name":" -b ", "type":"Flag", "label":"Show line numbers", "visible":true, "value": true, "validator":{ "name":"", "required":true }, "order":1 } ] } ], Grouping of options, panels. Parameters, options to set

18. >cat –b filename.txt >out.txt {... "output":[ { "id":"catout", "name":"", "type":"file", "multiplicity":"single", "order":3 "switch":" > " } ] }

19. For R? Not much different…

20. >Rscript --env script.R [options] Rscript --env script.R [options] [options]

21. Single site Tool Metadata { "location" : "/usr/bin/Rscript", "name" : "singleSite", "type" : "executable", "version" : 1.0, "description" : "Single Site analysis for breeding platform", "attribution" : "Graham McLaren (g.mclaren@cgiar.org)" }

22. singleSite.R data_1<-read.csv("one_site.csv", header=TRUE, na.strings= ".") #1 (index<-matrix(colnames(data_1),nrow=1)) LOC<-2 REP<-9 BLK<-6 ENTRY<-1 GY<-20 COVA<-c(15,18,23) base<-c(LOC,REP,BLK,ENTRY,GY) base_1<-data_1[,base] colnames(base_1)<-c("LOC","REP","BLK","ENTRY","GY")

23. singleSite.R args<-commandArgs(trailingOnly=TRUE) data_1<-read.csv(args[1], header=TRUE, na.strings= ".") #1 (index<-matrix(colnames(data_1),nrow=1)) LOC<-args[2] REP<-args[3] BLK<-args[4] ENTRY<-args[5] GY<-args[6] COVA<-args[-1:-6]

24. singleSite.R Rscript --env script.R [options] args[1] args[2] args[3] args[4] …

25. "input" : { "id" : "dataTable", "multiplicity" : "single", "name" : "Select the data table to analyze", "order" : 1, "switch" : "", "type" : "file" }, "input" : { "id" : "dataTable", "multiplicity" : "single", "name" : "Select the data table to analyze", "order" : 1, "switch" : "", "type" : "file" },

26. "groups" : { "id" : "", "name" : "Factors columns", "type" : "step", "properties" : [{ { "id" : "loc", "label" : "Location (LOC)", "name" : "", "order" : 2, "type" : "Number", "value" : "", "visible" : true, "validator" : { "name" : "", "required" : true, "rules" : { "IntAbove" : 0 } }, { "order" : 3, "id" : "rep", "label" : "Replicate (REP)", "name" : "", "type" : "Number", "value" : "", "visible" : true, "validator" : { "required" : true, "name" : "", "rules" : { "IntAbove" : 0 } }, { "order" : 4, "id" : "entry", "label" : "Treatment (ENTRY)", "name" : "", "type" : "Number", "value" : "", "visible" : true, "validator" : { "required" : true, "name" : "", "rules" : { "IntAbove" : 0 } }, { "order" : 5, "id" : "gy", "label" : "Response (GY)", "name" : "", "type" : "Number", "value" : "", "visible" : true, "validator" : { "required" : true, "name" : "", "rules" : { "IntAbove" : 0 } }, { "order" : 6, "id" : "cov", "label" : "Covariables (COVA)", "name" : "", "type" : "selectedHeadersList", "value" : "", "visible" : true, "validator" : { "required" : false, "name" : "" } }] },

27. args[1] args[2] args[3] args[4] … singleSite.R Rscript --env script.R [options]

28. "groups" : { "id" : "", "name" : "", "type" : "step", "properties" : [{ "order" : 1, "id" : "env", "label" : "", "name" : " --vanilla ", "type" : "flag", "value" : true, "visible" : false }, { "id" : "script", "label" : "", "name" : "", "order" : 2, "type" : "Text", "value" : "singleSite.R", "visible" : false } }],

29. "input" : { "id" : "dataTable", "multiplicity" : "single", "name" : "Select the data table to analyze", "order" : 3, "switch" : "", "type" : "file" }, "input" : { "id" : "dataTable", "multiplicity" : "single", "name" : "Select the data table to analyze", "order" : 3, "switch" : "", "type" : "file" },

30. { "id" : "", "name" : "Factors columns", "type" : "step", "properties" : [{ { "id" : "loc", "label" : "Location (LOC)", "name" : "", "order" : 4, "type" : "Number", "value" : "", "visible" : true, "validator" : { "name" : "", "required" : true, "rules" : { "IntAbove" : 0 } }, { "order" : 5, "id" : "rep", "label" : "Replicate (REP)", "name" : "", "type" : "Number", "value" : "", "visible" : true, "validator" : { "required" : true, "name" : "", "rules" : { "IntAbove" : 0 } }, { "order" : 4, "id" : "entry", "label" : "Treatment (ENTRY)", "name" : "", "type" : "Number", "value" : "", "visible" : true, "validator" : { "required" : true, "name" : "", "rules" : { "IntAbove" : 0 } }, { "order" : 5, "id" : "gy", "label" : "Response (GY)", "name" : "", "type" : "Number", "value" : "", "visible" : true, "validator" : { "required" : true, "name" : "", "rules" : { "IntAbove" : 0 } }, { "order" : 6, "id" : "cov", "label" : "Covariables (COVA)", "name" : "", "type" : "selectedHeadersList", "value" : "", "visible" : true, "validator" : { "required" : false, "name" : "" } }] },

31. Analyses How to put tools together

32. …JSON metadata

34. >cat { "analysis_id":"", "analysis_name":"File Read and Display", "type":"cat", "description":"Reads a file and prints to an output file", "steps":[ { "name":"cat", "description":"Reads a file and prints to standard output", "template_id":"", "config":{ "bflag":true } }, ], } From Tool metadata From Interface metadata

35. >cat –b filename.txt | grep cat -b grep Filename.txt

36. { "analysis_id":"", "analysis_name":"File Read and Display", "type":"Concatenate and print a file", "description":"Reads a file and prints to standard output", "steps":[ { "name":"cat", "description":"Reads a file and prints to standard output", "template_id":"", "config":{ } }, { "name":"grep_search", "description":"Searches within a file for supplied text", "template_id":"", "config":{ } }, ]

37. "mapping":[ { "source_step":"cat", "target_step":"grep_search", "map":{ "catout":"grepin" } }, ]

38. Complete material & tutorials at: http://www.iplantcollaborative.org https://pods.iplantcollaborative.org/wiki/display/docs/Analysis+Authoring+Tutorial https://pods.iplantcollaborative.org/wiki/display/docs/Tool+Integration+Tutorial