1. 1 Applied CyberInfrastructure Concepts ISTA 420/520 Fall 2013 1 Nirav Merchant (nirav@email.arizona.edu) Bio Computing & iPlant Collaborative Eric Lyons (ericlyons@email.arizona.edu) Plant Sciences & iPlant Collaborative University of Arizona http://goo.gl/p4j3mhttp://goo.gl/p4j3m or https://sites.google.com/site/appliedciconcepts/ Will Computers Crash Genomics? Science Vol 331 Feb 2011

2. Topic Coverage  Some pointers for your midterm  What are my choices for data transfer methods  Journey of data and gotcha  Data transfer != Data Management != Storage  iRODS hands on

3. Mid term  Consider Tyson as your client/customer and you are trying to win his business (and confidence)  Make it easy for him to use it, provide the instructions (user manual) for him to use it.  Include things like faq, gotcha. Expect that he is a new user with no idea about futuregrid, UA HPC etc. so be through and minimize jargon, explain things as step  The wiki page should be laid out such that he would want to hire you as his data team !  Your report should be well laid out and to the point  To run your job across distributed systems is good and recommended (use high port numbers 50000 and above )

4. How to transfer..and gotcha  http://moo.nac.uci.edu/~hjm/HOWTO_move_d ata.html http://moo.nac.uci.edu/~hjm/HOWTO_move_d ata.html  Network, disks, CPU and such  Some tools for bench marking and estimating – Measurement lab (NDT) http://www.measurementlab.net/ http://www.measurementlab.net/ – DDCopy http://monalisa.cern.ch/FDT/http://monalisa.cern.ch/FDT/  Bits and Bytes (Mb V/s MB)  http://www.wu.ece.ufl.edu/links/dataRate/Data MeasurementChart.html http://www.wu.ece.ufl.edu/links/dataRate/Data MeasurementChart.html

5. iPlant Data Store Free Your Data Different Users, Different Access Needs: One Data Store Different Users, Different Access Needs: One Data Store

6. The journey of data to iDS and challenges along the way ! Hard Drive Network card Building network Campus network Internet UA/FG network iDS Network card Hard Drive http://en.wikipedia.org/wiki/List_of_device_bandwidths Check: USB, HDD, Network capabilities

7. Basic Exercise  Lets learn about icommands  https://pods.iplantcollaborative.org/wiki/display/DS/Usi ng+iCommands https://pods.iplantcollaborative.org/wiki/display/DS/Usi ng+iCommands  Lets connect to UA HPC and use irods (hint module load)  TRANSFERING LARGE DATA FROM LAPTOP ON WIRELESS IS A BAD IDEA (steps below are from UA HPC after you ssh there)  Create a 100MB file (hint use dd and search for it) on UA HPC  Put it into iPlant data store (hint what icommand to use ?)  Log into http://de.iplantcollaborative.org and browse you data, share it with someonehttp://de.iplantcollaborative.org  Connect to future grid machine and get the same file

8. Basic exercise  Exploring special commands like imeta inc  (you will be better of getting icommands from iPlants wiki)