Research data management using Globus ESIP Summer Meeting 2015 Rachana Ananthakrishnan University of Chicago

Slides:



Advertisements
Similar presentations
5/30/2012. Provides a method for finding services/data on the Exchange Network – discover data. Supports User Friendly Tools Can automatically collect.
Advertisements

An Introduction to Repositories Thornton Staples Director of Community Strategy and Alliances Director of the Fedora Project.
GTS MetaData Generation data GTS data bases GTS Switch Volume C1 Central Support Office Information Classes white-list Metadata Synchronization.
Author - Title- Date - n° 1 GDMP The European DataGrid Project Team
Collaboration on Large Datasets using Globus Rachana Ananthakrishnan University of Chicago.
Javier Díaz, Alejandra Schiavoni, Ana Paola Amadeo, M. Emilia Charnelli Computer Science School National University of La Plata - Argentina Extending.
GridFTP: File Transfer Protocol in Grid Computing Networks
A Computation Management Agent for Multi-Institutional Grids
1 Archiving Workflow between a Local Repository and the National Library Archive Experiences from the DiVA Project Eva Müller, Peter Hansson, Uwe Klosa,
GGF Toronto Spitfire A Relational DB Service for the Grid Peter Z. Kunszt European DataGrid Data Management CERN Database Group.
NextGRID & OGSA Data Architectures: Example Scenarios Stephen Davey, NeSC, UK ISSGC06 Summer School, Ischia, Italy 12 th July 2006.
Robust Tools for Archiving and Preserving Digital Data Joseph JaJa, Mike Smorul, and Mike McGann Institute for Advanced Computer Studies Department of.
DataGrid Kimmo Soikkeli Ilkka Sormunen. What is DataGrid? DataGrid is a project that aims to enable access to geographically distributed computing power.
Tools and Services for the Long Term Preservation and Access of Digital Archives Joseph JaJa, Mike Smorul, and Sangchul Song Institute for Advanced Computer.
Enterprise Search With SharePoint Portal Server V2 Steve Tullis, Program Manager, Business Portal Group 3/5/2003.
© 2004, The Trustees of Indiana University 1 OneStart Workflow Basics Brian McGough, Manager, Systems Integration, UITS Ryan Kirkendall, Lead Developer.
UMIACS PAWN, LPE, and GRASP data grids Mike Smorul.
GT4 Introductory and Advanced Practicals Rachana Ananthakrishnan, Charles Bacon, Lisa Childers Argonne National Laboratory University of Chicago.
Archival Prototypes and Lessons Learned Mike Smorul UMIACS.
Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.
System Design/Implementation and Support for Build 2 PDS Management Council Face-to-Face Mountain View, CA Nov 30 - Dec 1, 2011 Sean Hardman.
Databases & Data Warehouses Chapter 3 Database Processing.
Architecture of information systems Document managment system Peter Záhorák.
Publishing Digital Content to a LOR Publishing Digital Content to a LOR 1.
The Data Replication Service Ann Chervenak Robert Schuler USC Information Sciences Institute.
Module 5: Managing Public Folders. Overview Managing Public Folder Data Managing Network Access to Public Folders Publishing an Outlook 2003 Form Discussion:
A Metadata Catalog Service for Data Intensive Applications Presented by Chin-Yi Tsai.
1 School of Computer, National University of Defense Technology A Profile on the Grid Data Engine (GridDaEn) Xiao Nong
Eric Holtel.  Introduction  Project Description  Demonstration  Deliverables  Conclusion.
Preserving Digital Culture: Tools & Strategies for Building Web Archives : Tools and Strategies for Building Web Archives Internet Librarian 2009 Tracy.
IT 456 Seminar 5 Dr Jeffrey A Robinson. Overview of Course Week 1 – Introduction Week 2 – Installation of SQL and management Tools Week 3 - Creating and.
1 Schema Registries Steven Hughes, Lou Reich, Dan Crichton NASA 21 October 2015.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Data Replication Service Sandeep Chandra GEON Systems Group San Diego Supercomputer Center.
Shannon Hastings Multiscale Computing Laboratory Department of Biomedical Informatics.
Data Publication and Quality Control Procedure for CMIP5 / IPCC-AR5 Data WDC Climate / DKRZ:
1 24 September BREAKOUT :30 1)Review of Metadata Standards Directory (DCC version and GitHub) 2)Introduction of Metadata Standards Catalog.
Copenhagen, 7 June 2006 Toolkit update and maintenance Anton Cupcea Finsiel Romania.
Web: Minimal Metadata for Data Services Through DIALOGUE Neil Chue Hong AHM2007.
Globus Replica Management Bill Allcock, ANL PPDG Meeting at SLAC 20 Sep 2000.
Oracle's Distributed Database Bora Yasa. Definition A Distributed Database is a set of databases stored on multiple computers at different locations and.
The Replica Location Service The Globus Project™ And The DataGrid Project Copyright (c) 2002 University of Chicago and The University of Southern California.
CLASS Information Management Presented at NOAATECH Conference 2006 Presented by Pat Schafer (CLASS-WV Development Lead)
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Construction of e-science Environment.
May 6, 2002Earth System Grid - Williams The Earth System Grid Presented by Dean N. Williams PI’s: Ian Foster (ANL); Don Middleton (NCAR); and Dean Williams.
Replica Management Kelly Clynes. Agenda Grid Computing Globus Toolkit What is Replica Management Replica Management in Globus Replica Management Catalog.
Globus online Software-as-a-Service for Research Data Management Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National.
1 Overall Architectural Design of the Earth System Grid.
The library is open Mobile Applications Russian-IUG November 2015 Tomsk, Russia Nabil Saadallah Manager Business Development.
Globus and ESGF Rachana Ananthakrishnan University of Chicago
Globus Publish Lighting Talk Ben Blaiszik, Kyle Chard
LLNL-PRES-XXXXXX This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344.
2-Hop TorrentSmell A distributed tracking algorithm name:Raynor Vliegendhart date:July 10, 2009 event:Tribler Dev Meeting.
Partnerships in Innovation: Serving a Networked Nation Grid Technologies: Foundations for Preservation Environments Portals for managing user interactions.
Globus.org/genomics Globus Galaxies Science Gateways as a Service Ravi K Madduri, University of Chicago and Argonne National Laboratory
The OAIS model SEEDS meeting May 5 th, 2015, Lausanne Bojana Tasic.
Metadata Organization and Management for Globalization of Data Access with Michał Wrzeszcz, Krzysztof Trzepla, Rafał Słota, Konrad Zemek, Tomasz Lichoń,
Matt Goldner Product & Technology Advocate Mela Kircher Product Manager WorldCat Local Metasearch 13 November 2009.
Globus online Delivering a scalable service Steve Tuecke Computation Institute University of Chicago and Argonne National Laboratory.
ETERE A Cloud Archive System. Cloud Goals Create a distributed repository of AV content Allows distributed users to access.
Ian Foster Ben Blaiszik Kyle Chard, Rachana Ananthakrishnan, Steven Tuecke, UChicago Michael Ondrejcek,
Using Git with collaboration, code review, and code management for open source and private projects. & Using Terminal to create, and push commits to repositories.
Simplifying Large-Scale Data Movement with Globus Steve Tuecke Deputy Director, Computation Institute University of Chicago & Argonne National Laboratory.
International Planetary Data Alliance Registry Project Update September 16, 2011.
AMSA TO 4 Advanced Technology for Sensor Clouds 09 May 2012 Anabas Inc. Indiana University.
TOWARDS AN ARCHITECTURE FOR NATIONAL DATA SERVICES Ian Foster Director, Computation Institute Argonne National Laboratory & The University of
Data mining in web applications
Enhancements to Galaxy for delivering on NIH Commons
Software infrastructure for a National Research Platform
Joseph JaJa, Mike Smorul, and Sangchul Song
Features Overview.
Presentation transcript:

Research data management using Globus ESIP Summer Meeting 2015 Rachana Ananthakrishnan University of Chicago

Globus delivers … … data transfer, sharing, publication, and discovery … on storage chosen by you 2

Reliable, secure, high-performance file transfer and replication “Fire-and-forget” transfers Automatic fault recovery Seamless security integration Powerful GUI and APIs Data Source Data Source Data Destination Data Destination User initiates transfer request 1 1 Globus moves and replicates files 2 2 Globus notifies user 3 3

Transfer Files

Transfer Options

Simple, secure sharing off existing storage systems Data Source Data Source User A selects file(s) to share, selects user or group, and sets permissions 1 1 Globus tracks shared files; no need to move files to cloud storage! 2 2 User B logs in to Globus and accesses shared file 3 3 Easily share large data with any user or group No cloud storage required

Share files/folders

Manage permissions 8

Globus Platform-as-a-Service Identity, Group, Profile Management Services … … Sharing Service Transfer Service Globus Toolkit Globus APIs Globus Connect

Curated publication of data, with relevant metadata for discovery Identify Describe Curate Verify Access Preserve Researcher assembles data set; describes it using metadata (Dublin core and domain-specific) 1 1 Peers, public search and discover data sets; transfer using Globus 3 3 Published Data Store Published Data Store Curator reviews and approves; data set published on campus or other storage 2 2 Metadata

Choose a collection A collection is created for each Case

Add description Metadata and ESGF node to store data

Assembles files to publish Identify files to publish and transfer to ESGF data node

Status updates Manage remote metadata extraction, generation of THREDDS catalogs and push to ESGF search index

Thank you to our sponsors! U.S. DEPARTMENT OF ENERGY 15