Managing Data with iPlant Introduction to Uploading, Downloading, Sharing, and Metadata in the Data Store.

Slides:



Advertisements
Similar presentations
Microsoft Office Illustrated Fundamentals Unit C: Getting Started with Unit C: Getting Started with Microsoft Office 2010 Microsoft Office 2010.
Advertisements

An Introduction to Using
Downloading, Installing, and Working with Dropbox.
Center Accreditation Online System
SOFTWARE PRESENTATION ODMS (OPEN SOURCE DOCUMENT MANAGEMENT SYSTEM)
1 of 6 Parts of Your Notebook Below is a graphic overview of the different parts of a OneNote 2007 notebook. Microsoft ® OneNote ® 2007 notebooks are digital.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment Chapter 8: Implementing and Managing Printers.
Chapter Accreditation Online System Usage Tutorial Department of Member Relations & Grants National Children’s Alliance.
Application Process USAJOBS – Application Manager USA STAFFING ® —OPM’S AUTOMATED HIRING TOOL FOR FEDERAL AGENCIES.
1 Box Overview Session Getting Started on Box. 2 Agenda What is Box? Box Basics Live Q&A.
Working with SharePoint Document Libraries. What are document libraries? Document libraries are collections of files that you can share with team members.
The iPlant Collaborative Community Cyberinfrastructure for Life Science Tools and Services Workshop Discovery Environment Overview.
1 iPlant Data Store (iDS) Supporting the Lifecycle of Data Nirav Merchant 1.
Use Watch folders to automatically add PDFs to Mendeley Desktop. When you place a document in a watched folder, it will be automatically added to Mendeley.
Google Chrome Your Customized Google Buddy April 2012 John Riley and Denise Tate-Kuhler.
With Windows 7 Comprehensive© 2012 Pearson Education, Inc. Publishing as Prentice Hall1 PowerPoint Presentation to Accompany GO! with Windows 7 Comprehensive.
+ Working in Your CCE Online Course Site. + Structure of CCE Online Course Sites CCE online courses use the document sharing and collaboration features.
Josh Probert – Yankee A Prototype based on Sierra’s SRS.
Customized cloud platform for computing on your terms !
Presented by Chad Kafka This Month’s Topic: Wikispaces Advanced Today’s session is an introduction to what a WIKI is and how they can be used in education.
Extending the Discovery Environment: Tool Integration and Customization.
Delight QuickBooks Online Banking Internal Support Training QuickBooks Windows 2009/2010 Online Banking.
Home Media Network Hard Drive Training for Update to 2.0 By Erik Collett Revised for Firmware Update.
Download Dropbox Download should start immediately Save download file:
An Introduction to Designing and Executing Workflows with Taverna Katy Wolstencroft University of Manchester.
The iPlant Collaborative Community Cyberinfrastructure for Life Science Tools and Services Workshop iCommands and Other Data Store Resources.
Downloading and Installing Autodesk Revit 2016
2008 Adobe Systems Incorporated. All Rights Reserved. Getting Started with Adobe Presenter Modified by C. Candace Chou.
An Introduction to Designing and Executing Workflows with Taverna Aleksandra Pawlik materials by: Katy Wolstencroft University of Manchester.
The iPlant Collaborative Community Cyberinfrastructure for Life Science Tools and Services Workshop Discovery Environment Overview.
Downloading and Installing Autodesk Inventor Professional 2015 This is a 4 step process 1.Register with the Autodesk Student Community 2.Downloading the.
The iPlant Collaborative Community Cyberinfrastructure for Life Science Tools and Services Workshop iPlant Data Store.
Tour Overview Introduction Collage Basics Collage Basics (Templates and Tools) Computer Configuration Bookmark Collage Getting Started Tour Collage Terminology.
The iPlant Collaborative Community Cyberinfrastructure for Life Science Tools and Services Workshop Discovery Environment Overview.
IPlant Collaborative Hands-on Cyberinfrastructure Workshop - Part 1 R. Walls University of Arizona Biodiversity Information Standards (TDWG) Sep. 28, 2015,
THE 3M CLOUD LIBRARY AND YOUR KINDLE FIRE How to download 3M books to your Kindle Fire/HD Made possible by a grant from the Samuel P. Hunt foundation.
The iPlant Collaborative Community Cyberinfrastructure for Life Science Tools and Services Workshop iPlant Data Store – Managing Your ‘Big’ Data.
Build an Automated Workflow Visual Workflow Creator Discovery Environment.
Getting to Know Your Desktop Icons, Taskbar, Workspace, Window, Notifications, Start and Search.
CyVerse-enabled NCBI Sequence Read Archive (SRA) Submission Pipeline
IPlant Collaborative Tools and Services Workshop Overview of the iPlant Discovery Environment Sriram Srinivasan.
ITS Lunch & Learn November 13, What is Office 365? Office 365 is Microsoft’s software as a service offering. It includes hosted and calendaring.
Lindsey Velez, Director of Instructional Technology Single Sign-On One Click.
Transforming Science Through Data-driven Discovery Tools and Services Workshop Data Store Overview.
CyVerse Workshop Discovery Environment Overview. Welcome to the Discovery Environment A Simple Interface to Hundreds of Bioinformatics Apps, Powerful.
Transforming Science Through Data-driven Discovery Tools and Services Workshop Data Store – Managing your ‘Big’ Data Joslynn Lee, Ph.D. – Data Science.
THE NEW MOBILE WORKSPACE Enable Business Applications on Mobile Devices hopTo Work “I am amazed to see how easily hopTo transforms the user interface of.
Special Education Teachers and Speech Language Pathologist Effective Technology Tools By: Beth Fulks, June 23, 2014.
Classroom Wiki Tutorial EDIC 763 Instructional Design Fall 2011 Aysha Bajabaa Dr. Gary Whitt 1 NextBack.
CyVerse Data Store Managing Your ‘Big’ Data. Welcome to the Data Store Manage and share your data across all CyVerse platforms.
How to Apply PDF in Flipbook on Website. Description If you are finding solution for applying PDF in flipbook mode on website, and adding multimedia items.
Jacynthe Touchette, MSI JGH Health Sciences Library
Core ELN Training: Office Web Apps (OWA)
File Management in the Cloud
Objectives Create a folder in Google Drive.
Delicious Social Bookmarking
New Box Web Experience Inventory of changes.
CyVerse Discovery Environment
For Beginners Mike Buhmann Reference Librarian.
Tools and Services Workshop
Data uploading and sharing with CyVerse
Instructor Name Instructor Title Library Name
OneDrive for Business User Guide
SRA Submission Pipeline
An Introduction to Using
Cloud Connect Seamlessly
ICT Word Processing Lesson 5: Revising and Collaborating on Documents
Download from Zotero Home Page
Zotero 5.0 and Juris-M: Reference Management for Law
Microsoft Office Illustrated Fundamentals
Presentation transcript:

Managing Data with iPlant Introduction to Uploading, Downloading, Sharing, and Metadata in the Data Store

Background on the iPlant Data Store iPlant Learning Center provides quick tutorials, slides, and documentation for everything you will see here iPlant Learning Center Backbone of the iPlant CI Connects to all iPlant services Appropriate for any (research) file types of any size Cloud Based, Backed Up, Initial 100 GB expandable to 1 TB Built on iRODS Folder = Collection Other than that, you don’t have to think about this if you don’t want to Demo 1 – Navigating the Data Store in the Discovery Environment

Upload and Download In the Discovery Environment  ‘Simple’, for small files (~ 5 files, <1.9 GB)  ‘Bulk’, for larger files and folders (<10GB)  Import from URL (no size limit) Advantage + Disadvantage - Covers most upload/download sharing needs Point and Click Some size/speed limitations Demo 2 – Transferring Data from the Discovery Environment

Tips Spaces /Special Characters Many software packages are sensitive to spaces in file names and/or the special characters below. Users may wish to rename uploaded files before using them in an analysis. Good advice for any transfer method. ~` $ % ^& *()+ = {}[]|\:;"'<>,?/

Bulk Transfers Requires Java 6 or later be enabled in your browser ( Not currently compatible with Google Chrome Window and web browser must remain open and active until the transfer is complete Import from URL Monitor Notifications to check that the URL import has been submitted – you will receive a notification when import is complete Tips

iDrop Desktop  Drag and Drop files and folders  File sizes up to your total allocation  Fast transfers  Synchronize folders with Data Store  Download and Installation Instructions Download and Installation Instructions  Can demo installation Advantage + Disadvantage - Upload/download large file sizes and numbers of files Sharing and permission features more complex Demo 3 – Transferring Data with iDrop Desktop

iDrop Requires Java 6 or later ( At the bottom of the iDrop window you can monitor, pause, and restart transfers. You can also view additional details by clicking Manage. When iDrop Desktop is open/running there will be an icon in your system tray or menu along with other background programs (e.g. Wi-Fi, Bluetooth, etc.). You need to close iDrop from this icon to completely close the program. Tips

iCommands Advantage + Disadvantage - Customizability Requires at least some command line expertise  Ability to script and automate  Access from terminal/server  Can resume transfers  Download and installation instructions Download and installation instructions  Can demo installation Demo 4 – Accessing the Data Store with iCommands

Tips -f force - overwrite local files -P output the progress of the download. -r recursive - retrieve subcollections (directories) -T renew socket after 10 min.(use with large files) -V Very verbose Useful iget / iput options:

Sharing Files in the Data Store Discovery Environment Sharing Sharing via Public Link Share files/folders instantly Control access permissions Manage sharing between collaborators No iPlant account required Limited to individual files URLs are public (less secure, can revoke) 2 Easy ways to share data within the Discovery Environment Demo 5 – Data Sharing in the DE

Tips When sharing via the Discovery Environment, use the following chart to decide the permissions you wish to grant: PermissionReadDownloadMetadataInfo TypesRenameMoveDelete Read Write Own

Need help? iPlant Learning Center provides quick tutorials, slides, and documentation for everything you will see here iPlant Learning Center

Analyses in the Discovery Environment Using Bioinformatics Apps

Background on the iPlant Discovery Environment iPlant Learning Center provides quick tutorials, slides, and documentation for everything you will see here iPlant Learning Center So far we have mainly looked at the Data Tab DE is also a powerful interface for Apps and Analyses This is where scalability and extensibility really come into play Can customize and create new apps Seamlessly integrated with high-throughput computing Apps are linked to iPlant wiki for documentation Demo 1 – Apps and analyses in the DE

Tips Mark an App as a favorite and it will appear in your workspace. Use the Apps menu to: Rate Apps to provide feedback Click the info icon to see the user manual for an App Re-launch a job by clicking on the App name in the Analyses menu. The App will re-launch populated with the last parameters used, giving you the option to alter the settings you want.

Tips View a file containing metadata (settings, etc.) connected to your analysis. Select the a job and click View Parameters.

Tips Can customize tools in the DE (simplifying here) App installation / modification involves happens at two levels: Apps (and dependencies) are installed on DE Cluster (done by iPlant support) DE Interface (created by App integrator/user) and published to DE Detailed instructions with videos, manuals, documentation in Learning CenterLearning Center

Viewing and Editing Metadata In the DE User metadata stored AVUs Attribute – Value – Unit Template-based metadata Can view and edit from with iCommands (tomorrow) Demo 2 – Metadata in the DE

Tips Currently can only use one template at a time Can create custom metadata templates Metadata in the DE

Need help? iPlant Learning Center provides quick tutorials, slides, and documentation for everything you will see here iPlant Learning Center

Syncing Folders With iDrop iPlant Learning CenteriPlant Learning Center provides quick tutorials, slides, and documentation for everything you will see here

iDrop Desktop Synchronization 1.Click on Settings 2.Click on the Synchronization tab 3.Click New to start a new synchronization 4.Enter a name (e.g. Project 1 sequence data) 5.Click Choose Local Folder to select a folder 6.Select a synchronization mode 7.Select a frequency for synchronization 8.Click Choose iRODS Folder to select the location to synch to 9.Click Update to save your synchronization Demo – DropBox Sync

Tips  Choose one of three methods to synchronize files: Local > iRODS – files will copied at synch to Data Store Local < iRODS – files in Data Store copied to local computer at synch Local <> iRODS – files in Data Store and local computer both synched If syncing to a DropBox folder, Dropbox must be paused during set up

Need help? iPlant Learning Center provides quick tutorials, slides, and documentation for everything you will see here iPlant Learning Center

Advanced Topics Searching in the DE, iCommands and Metadata, Data Commons Plans and Progress

Searching in the Discovery Environment Demo 1 – Basic and advanced searching Basic search bar lets you search all files and folders where you have permission Advanced search features allows searching based on metadata, permissions, and share status

iCommands to view and edit metadata Demo 2 – imeta commands Ability to interact with metadata at the command line Already installed as part of iCommands Documentation is a little wimpy Try Here and HereHere

Tips At the moment, metadata added via template is not available from the command line

Data, Project, Research Management In the Data Commons Tools and for sharing, managing, and publishing data A home for high-value public dataset to be used with iPlant analysis tools A way to manage projects Metadata templates and workflows for common analysis types Working hard to lay the groundwork, with development starting early 2015

Data, Project, Research Management In the Data Commons Additional layer on top of the Data Store for Users to publish packages of data and metadata to the Data Commons and supported external repositories with appropriate long-term identifiers and licenses Data will be static, searchable, discoverable, and linked to external repositories Based on Data Strategy, current CI, Developer input, recognized need for additional components to support Data Commons effort

Data Commons Development Plans

Goal: Data models and workflows for the entire data lifecycle specimen collection analysis project creation publication

Interface Mockups are in Review Demo 3 – Data Commons Interface Mockups

Metadata Templates for G2F Open discussion now that we are all updated on G2F and iPlant infrastructure Project Interface needs for G2F? Metadata needs for G2F? Other needs? Perhaps start here?here

Thanks! iPlant Learning Center provides quick tutorials, slides, and documentation iPlant Learning Center