3 September 2004NVO Coordination Meeting1 Grid-Technologies NVO and the Grid Reagan W. Moore George Kremenek Leesa Brieger Ewa Deelman Roy Williams John.

Slides:



Advertisements
Similar presentations
May IVOA Interop Meeting1 STScI/JHU Registry Status Gretchen Greene Wil OMullane T HE US N ATIONAL V IRTUAL O BSERVATORY.
Advertisements


San Diego Supercomputer Center & National Partnership for Advance Computational Infrastructure Storage Resource Broker Reagan W. Moore San Diego Supercomputer.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Data Grids for Collection Federation Reagan W. Moore University.
VO Standards – Catalog Access Tamás Budavári Johns Hopkins University.
8 September 2008NVO Summer School 2008 – Santa Fe1 Publishing Data and Services to the VO Ray Plante Gretchen Greene T HE US N ATIONAL V IRTUAL O BSERVATORY.
LEAD Portal: a TeraGrid Gateway and Application Service Architecture Marcus Christie and Suresh Marru Indiana University LEAD Project (
© 2007 Open Grid Forum Data Management Challenge - The View from OGF OGF22 – February 28, 2008 Cambridge, MA, USA Erwin Laure David E. Martin Data Area.
A PPARC funded project AstroGrid Framework Consortium meeting, Dec 14-15, 2004 Edinburgh Tony Linde Programme Manager.
A PPARC funded project The Grid Data Warehouse Description of prototype work in progress by AstroGrid. Access-Grid lecture to Universities of Leeds and.
Distributed Systems basics
Data Grid: Storage Resource Broker Mike Smorul. SRB Overview Developed at San Diego Supercomputing Center. Provides the abstraction mechanisms needed.
NATIONAL PARTNERSHIP FOR ADVANCED COMPUTATIONAL INFRASTRUCTURE SAN DIEGO SUPERCOMPUTER CENTER Particle Physics Data Grid PPDG Data Handling System Reagan.
14 October 2003ADASS 2003 – Strasbourg1 Resource Registries for the Virtual Observatory R.Plante (NCSA), G. Greene (STScI), R. Hanisch (STScI), T. McGlynn.
October 12, 2003ADASS NVO Tutorial1 How-To Implement Cone and SIA Services Gretchen Greene Space Telescope Science.
GGF-17 Astro Workshop Preservation Environment Working Group Officers: Bruce Barkstrom (NASA Langley) Reagan Moore (SDSC) Goals  Demonstrate.
The Globus Toolkit Gary Jackson. Introduction The Globus Toolkit is a product of the Globus Alliance ( It is middleware for developing.
Data Grids: Globus vs SRB. Maturity SRB  Older code base  Widely accepted across multiple communities  Core components are tightly integrated Globus.
Virtual Observatory Architecture Data Services Registry Services Compute Services Roy Williams Caltech US-VO co-director.
Robust Tools for Archiving and Preserving Digital Data Joseph JaJa, Mike Smorul, and Mike McGann Institute for Advanced Computer Studies Department of.
July 2004SC4DEVO -Caltech1 IVOA WebServices William O’Mullane The Johns Hopkins University T HE US N ATIONAL V IRTUAL O BSERVATORY.
Mike Smorul Saurabh Channan Digital Preservation and Archiving at the Institute for Advanced Computer Studies University of Maryland, College Park.
Pegasus: Mapping complex applications onto the Grid Ewa Deelman Center for Grid Technologies USC Information Sciences Institute.
UMIACS PAWN, LPE, and GRASP data grids Mike Smorul.
The Open Grid Service Architecture (OGSA) Standard for Grid Computing Prepared by: Haoliang Robin Yu.
An Astronomical Image Mosaic Service for the National Virtual Observatory
An Astronomical Image Mosaic Service for the National Virtual Observatory / ESTO.
Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.
Why Build Image Mosaics for Wide Area Surveys? An All-Sky 2MASS Mosaic Constructed on the TeraGrid A. C. Laity, G. B. Berriman, J. C. Good (IPAC, Caltech);
The Grid is a complex, distributed and heterogeneous execution environment. Running applications requires the knowledge of many grid services: users need.
National Partnership for Advanced Computational Infrastructure Digital Library Architecture Reagan Moore Chaitan Baru Amarnath Gupta George Kremenek Bertram.
San Diego Supercomputer CenterUniversity of California, San Diego Preservation Research Roadmap Reagan W. Moore San Diego Supercomputer Center
The Japanese Virtual Observatory (JVO) Yuji Shirasaki National Astronomical Observatory of Japan.
Data Management Kelly Clynes Caitlin Minteer. Agenda Globus Toolkit Basic Data Management Systems Overview of Data Management Data Movement Grid FTP Reliable.
OPEN GRID SERVICES ARCHITECTURE AND GLOBUS TOOLKIT 4
ARGONNE  CHICAGO Ian Foster Discussion Points l Maintaining the right balance between research and development l Maintaining focus vs. accepting broader.
GT Components. Globus Toolkit A “toolkit” of services and packages for creating the basic grid computing infrastructure Higher level tools added to this.
Virtual Observatory & LIGO Roy Williams California Institute of Technology.
Semantic Grid + Data Federation US National Virtual Observatory Roy Williams California Institute of Technology NVO co-director.
Production Data Grids SRB - iRODS Storage Resource Broker Reagan W. Moore
Pegasus: Mapping Scientific Workflows onto the Grid Ewa Deelman Center for Grid Technologies USC Information Sciences Institute.
Virtual Data Grid Architecture Ewa Deelman, Ian Foster, Carl Kesselman, Miron Livny.
Federation and Fusion of astronomical information Daniel Egret & Françoise Genova, CDS, Strasbourg Standards and tools for the Virtual Observatories.
Rule-Based Preservation Systems Reagan W. Moore Wayne Schroeder Mike Wan Arcot Rajasekar Richard Marciano {moore, schroede, mwan, sekar,
TeraGrid CTSS Plans and Status Dane Skow for Lee Liming and JP Navarro OSG Consortium Meeting 22 August, 2006.
National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Persistent Management of Distributed Data Reagan W. Moore.
National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Persistent Archive for the NSDL Reagan W. Moore Charlie Cowart.
Pegasus: Running Large-Scale Scientific Workflows on the TeraGrid Ewa Deelman USC Information Sciences Institute
Policy Based Data Management Data-Intensive Computing Distributed Collections Grid-Enabled Storage iRODS Reagan W. Moore 1.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
GGF-17 Astronomy Application Meeting Astro Working Group Officers: Nic Walton (Astrogrid, Cambridge) Guy Rixon (Astrogrid, Cambridge)
Pegasus: Mapping complex applications onto the Grid Ewa Deelman Center for Grid Technologies USC Information Sciences Institute.
GRIDS Center Middleware Overview Sandra Redman Information Technology and Systems Center and Information Technology Research Center National Space Science.
GEON2 and OpenEarth Framework (OEF) Bradley Wallet School of Geology and Geophysics, University of Oklahoma
12 Oct 2003VO Tutorial, ADASS Strasbourg, Data Access Layer (DAL) Tutorial Doug Tody, National Radio Astronomy Observatory T HE US N ATIONAL V IRTUAL.
Biomedical Informatics Research Network The Storage Resource Broker & Integration with NMI Middleware Arcot Rajasekar, BIRN-CC SDSC October 9th 2002 BIRN.
National Archives and Records Administration1 Integrated Rules Ordered Data System (“IRODS”) Technology Research: Digital Preservation Technology in a.
Rights Management for Shared Collections Storage Resource Broker Reagan W. Moore
Building Preservation Environments with Data Grid Technology Reagan W. Moore Presenter: Praveen Namburi.
Preservation Data Services Persistent Archive Research Group Reagan W. Moore October 1, 2003.
Grid Services for Digital Archive Tao-Sheng Chen Academia Sinica Computing Centre
Data Grids, Digital Libraries and Persistent Archives: An Integrated Approach to Publishing, Sharing and Archiving Data. Written By: R. Moore, A. Rajasekar,
MSSL Astrogrid Workshop
The Open Grid Service Architecture (OGSA) Standard for Grid Computing
Pegasus and Condor Gaurang Mehta, Ewa Deelman, Carl Kesselman, Karan Vahi Center For Grid Technologies USC/ISI.
Arcot Rajasekar Michael Wan Reagan Moore (sekar, mwan,
Google Sky.
Presentation transcript:

3 September 2004NVO Coordination Meeting1 Grid-Technologies NVO and the Grid Reagan W. Moore George Kremenek Leesa Brieger Ewa Deelman Roy Williams John Good Bruce Berriman

3 September 2004NVO Coordination Meeting2 Why Grids? Support for large-scale data processing Collection-based distributed data management –Replication of large collections onto the Teragrid Access to distributed resources –Distributed data caching –Workflow processing systems –Distributed job execution –Common authentication environment –Service life cycle management

3 September 2004NVO Coordination Meeting3 Grid Technologies Grid Portals - GridPort Workflow control pipelines - Chimera/Pegasus Job scheduling management - CondorG Job execution system - GRAM Data caching and replication - RLS Authentication system - GSI Large file data transport – GridFTP, RFT Metadata catalog - MCS, MCAT Collection management – SRB Database Access on the Grid: OGSA-DAI

3 September 2004NVO Coordination Meeting4 NVO Grid Components View 1 - resources Portals - web service interfaces to analysis procedures (cone search, VOTable catalog query, simple image access) Process management systems - data processing pipelines to create derived data products Data access layer - management of methods on data encoding formats Data grid - management of distributed collections. Provide logical name space for global persistent identifiers. Persistent data caches - interactive access to sky survey image collections (Grid Bricks) High performance disk caches - high speed access for bulk data analysis. Computational grid - access to distributed compute resources. Persistent archives - management of technology evolution.

3 September 2004NVO Coordination Meeting5 NVO Grid Components View 2 - data virtualization Data grids provide a standard access mechanism to image archives –Install data grid interface to an existing archive –Install NVO service on top of data grid NVO services ported to Data Grids –Image cutout –Mosaicing Integrate processing pipeline on top of Grid –Workflow composed from NVO services –Chimera and Pegasus

3 September 2004NVO Coordination Meeting6 NVO Grid Components View 3 - services NVO Services –Montage, Image cutout, VOTable Image processing pipelines –Rome Image archives –2MASS, DPOSS, SDSS, USNO-B, … Grid infrastructure –GSI –Chimera / Pegasus –CondorG –GRAM –RLS

3 September 2004NVO Coordination Meeting7 NVO Grid Components View 4 - services Globus2: Access to distributed compute resources SRB: Collection-based distributed data management NVO registry: Distributed semantic grid with OAI transport. It provides the kind of publication, searching, and capability information to enable a community to build a virtual organization of services. OpenSkyQuery and SIAP: These services are providing bulk data via SRB to Grid projects on Teragrid (Atlasmaker). They will become the basis of astronomical data delivery via OGSA services. OGSA: A specification only, but Globus3 and OGSAdotNet are implementations. Based on SOAP and web services. We can convert OpenSkyQuery to OGSA services. Grist: This new project at Caltech and JPL (if formally funded by NSF) will build a library of OGSA services for data mining in astronomy, with a dataflow mechanism to control them.

3 September 2004NVO Coordination Meeting8 Challenges Grid technology is rapidly evolving; activities in progress –GT3 - OGSA based services Compatibility with WSDL from the Semantic Web –GridFTP rewrite, protocol redesign –Chimera redesign (support collection access) –OGSA-DAI data access interface –Data Format Description Language –Grid File System design

3 September 2004NVO Coordination Meeting9 Next Steps Grid environment –OGSA based service support Grid enabled NVO services –Support access to data through data grid Demonstration of an analysis of an entire sky survey image archive –2MASS mosaic Integration of additional image archives into grid Grid enabled processing pipelines