GRID Perspectives at ESA/ESAC Science Archives and VO Team Computer Support Group ESAC, Villafranca del Castillo, Madrid, Spain.

Slides:

Advertisements

Similar presentations

VODA - A Sampo Project Johan Lindroos – CSC Scientific Computing Ltd, Finland Pekka Järveläinen – CSC Scientific Computing Ltd, Finland Richard Hook -

Advertisements

European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Aurélien Stébé AIDA WP3 - Curation September 2009, VOTech, Trieste Curation.

European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Aurélien Stébé DALToolKit Ingestor & Server September 2006, Moscow DALToolKit.

ESA/ESAC European Space Astronomy Center Villafranca del Castillo MADRID SPAIN IVOA Workshop 2004 Cambridge Mass Pedro OSUNA ESAC(VILSPA) proposals for.

European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Jesús Salgado SLAP Implementations Sep 2006, Moscow, Russia Simple Line.

Photometry Data Model| Jesus Salgado | ESAC | 19 May 2010 | IVOA Interop Victoria | Pag. 1/5 Photometry data model Jesus Salgado

SLAP: Simple Line Access Protocol v0.5

May 9, 2007Astro-RG, OGF201 Japanese Virtual Observatory and NaReGi Masatoshi Ohishi / NAOJ, Sokendai & NII /, &

CBPF J. Magnin LAFEX-CBPF. Outline What is the GRID ? Why GRID at CBPF ? What are our needs ? Status of GRID at CBPF.

INTRODUCTION TO SIMULATION WITH OMNET++ José Daniel García Sánchez ARCOS Group – University Carlos III of Madrid.

Beatriz Martinez - SAT CFA Team 2 nd CFA Implementation Review ESAC, Madrid, 06 Jun 2013 CLUSTER FINAL ARCHIVE (CFA) Future developments.

ESO-ESA Existing Activities Archives, Virtual Observatories and the Grid.

GRID Activities at ESAC Science Archives and Computer Engineering Unit Science Operations Department ESA/ESAC – Madrid, Spain.

IFIN-HH LHCB GRID Activities Eduard Pauna Radu Stoica.

Astronomical GRID Applications at ESAC Science Archives and Computer Engineering Unit Science Operations Department ESA/ESAC.

Aus-VO: Progress in the Australian Virtual Observatory Tara Murphy Australia Telescope National Facility.

ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University SCOTGRID: A PROTOTYPE TIER-2 CENTRE Steve Thorn Authors: A. Earl, P. Clark, S.

Virtual Observatory --Architecture and Specifications Chenzhou Cui Chinese Virtual Observatory (China-VO) National Astronomical Observatory of China.

A PPARC funded project AstroGrid: new technology for the virtual observatory SC2004 Pittsburgh, PA November 2004 Guy Rixon AstroGrid Technical Architect.

Planetary Science Archive (PSA) and PDS Introduction VEX / MEX Data Workshop 27 th June to 1 st July 2011 ESAC Dave Heather

ENTERPRISE COMPUTING QUIZ By: Lean F. Torida

Viewgraph 1 S. Ott Herschel Data Processing System NHSC Herschel Data Processing Workshop 26th August 2013 Overview of the Herschel Data Processing System.

Group Computing Strategy Introduction and BaBar Roger Barlow June 28 th 2005.

VO Sandpit, November 2009 e-Infrastructure to enable EO and Climate Science Dr Victoria Bennett Centre for Environmental Data Archival (CEDA)

1 Evolution of OSG to support virtualization and multi-core applications (Perspective of a Condor Guy) Dan Bradley University of Wisconsin Workshop on.

European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Jesús Salgado SLAP Implementations May 2007, Beijing, China Simple Line.

How to Adapt existing Archives to VO: the ISO and XMM-Newton cases Research and Scientific Support Department Science Operations.

Markus Dolensky, ESO Technical Lead The AVO Project Overview & Context ASTRO-WISE ((G)A)VO Meeting, Groningen, 06-May-2004 A number of slides are based.

European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Isa Barbarisi VOSpec, new functionalities Madrid, 6-7Oct 2005 New functionalities.

European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Jesús Salgado Spectroscopic lines in the VO context Mar 2007, ESAC, Madrid,

ESAVO/European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Isa Barbarisi Beijing, May 2007 VOSpec new functionalities.

European New HST & MMI Demo Nacho León María Arévalo Jonas Haase Jesús Salgado Deborah Baines Bruno Merín ESAC 20 January 2015.

The II SAS Testbed Site Jan Astalos - Institute of Informatics Slovak Academy of Sciences.

European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Jesús Salgado IVOA Interop meeting Strasbourg, May 2009 (1/12) SLAP v0.9.

European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Pedro OSUNA ESAC ADT Team VO-tech Cambridge 2004 VOSpec: A Tool to Handle.

CEOS WGISS-21 CNES GRID related R&D activities Anne JEAN-ANTOINE PICCOLO CEOS WGISS-21 – Budapest – 2006, 8-12 May.

European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Applications May 2006, Victoria, Canada VOQuest A tool.

Evolution of a High Performance Computing and Monitoring system onto the GRID for High Energy Experiments T.L. Hsieh, S. Hou, P.K. Teng Academia Sinica,

Science ESAC Cluster Final Archive Pedro Osuna Head of the Science Archives and VO Team Science Operations Department CAA-CFA Review Meeting.

WP 7 - JRA2 - Data Access Protocols and Data Models EuroVO-AIDA – Final review – 5 October 2010 Christophe Arviset (ESA) Work Package 7 - JRA2 Data Access.

21-jun-2009 IVOA Standards Pedro Osuna ESA-VO Project Science Archives and Computer Support Engineering Unit (SRE-OE) Science Operations Department (SRE-O)

Science Operations & Data Systems Division Research & Scientific Support Department IVOA Tutorial Strasbourg Pedro OSUNA How to adapt existing archives.

European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Jesús Salgado AIDA Tech. Meeting Strasbourg, March 2009 (1/8) WP7 Task.

ESAVO/European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Pedro Osuna ASVOWS Mar 2007 ESAC Astronomical Spectroscopy.

ESAVO/European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Pedro Osuna ASVOWS Mar 2007 ESAC Astronomical Spectroscopy.

Christophe Arviset Paolo Padovani EURO-VO DCA Review, EC, Brussels, 10/01/2008 page 1 DCA WP3 Report Support to take-up and implementation of the VObs.

Introduction to the VO ESAVO ESA/ESAC – Madrid, Spain.

State of the TCG Christophe Arviset For the TCG. TCG State of the TCG 6 December 2010 Christophe Arviset – TCG chair Page 2 Technical Coordination Group.

European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Pedro Osuna VOSpec Kyoto May 2005 VOSpec: A Tool to Handle VO-Compatible.

Christophe ARVISET EURO-VO AIDA Kick-Off, 17-18/03/2008, Strasbourg, page 1 ESAVO Project WP3 – SA1 Service activities in support of deployment of IVOA.

F. Genova, AstroNET meeting, Poitiers The Astrophysical Virtual Observatory.

ESA Scientific Archives and Virtual Observatory Systems Science Archives and VO Team Research and Scientific Support Department.

Western Tier 2 Site at SLAC Wei Yang US ATLAS Tier 2 Workshop Harvard University August 17-18, 2006.

Collaborative Tools for the Grid V.N Alexandrov S. Mehmood Hasan.

Claudio Grandi INFN Bologna Virtual Pools for Interactive Analysis and Software Development through an Integrated Cloud Environment Claudio Grandi (INFN.

INFN/IGI contributions Federated Clouds Task Force F2F meeting November 24, 2011, Amsterdam.

1 Workflows Access and Massage VO Data José Enrique Ruiz on behalf of the Wf4Ever Team IVOA INTEROP SPRING MEETING 2013 HEIDELBERG, MAY16th 2013.

Science Gateway- 13 th May Science Gateway Use Cases/Interfaces D. Sanchez, N. Neyroud.

Centre for Parallel Computing Tamas Kiss Centre for Parallel Computing A Distributed Rendering Service Tamas Kiss Centre for Parallel Computing Research.

Scientific Data Processing Portal and Heterogeneous Computing Resources at NRC “Kurchatov Institute” V. Aulov, D. Drizhuk, A. Klimentov, R. Mashinistov,

DCA WP3 Report Support to take-up and implementation of the VObs framework Christophe Arviset (ESA-VO) Paolo Padovani (ESO) Christophe Arviset (ESA),

Title of the Poster Supervised By: Prof.*********

Matt Lemons Nate Mayotte

Archiving of solar data Luis Sanchez Solar and Heliospheric Archive Scientist Research and Scientific Support Department.

Building a worldwide interoperable data infrastructure for astronomy: the International Virtual Observatory Alliance SciDataCon 2016, Denver, 13/Sept/2016.

IVOA Status Christophe Arviset

04/02/2019 The Use of Grid Technology in Large Scale Data Processing Collaboration Environments S.G. Ansari S.G. Ansari 04/02/2019.

IVOA – VOQL WG session ESAC Villafranca del Castillo Madrid, Spain Friday, October 7th, 9: :00 Yuji Shirasaki Maria Nieto-Santisteban VOQL-Madrid.

Presentation transcript:

GRID Perspectives at ESA/ESAC Science Archives and VO Team Computer Support Group ESAC, Villafranca del Castillo, Madrid, Spain

Science Operations & Data Systems Division Research & Scientific Support Department Christophe ARVISET Science Operations & Data Systems Division Research & Scientific Support Department IVOA Interop, Victoria, 17/05/2006, page 2 Main Goals of ESAC GRID activities Build a GRID at ESAC that can be used by all projects at ESAC Scientists to use GRID for cpu intensive tasks Support to Projects for Pipeline Data Processing Integration with other GRIDs ESTEC, ESRIN, UCM, others, …. Closer link to the VO

Science Operations & Data Systems Division Research & Scientific Support Department Christophe ARVISET Science Operations & Data Systems Division Research & Scientific Support Department IVOA Interop, Victoria, 17/05/2006, page 3 Grid Computing scigrid.esac.esa.int vogrid.esac.esa.int GT4

Science Operations & Data Systems Division Research & Scientific Support Department Christophe ARVISET Science Operations & Data Systems Division Research & Scientific Support Department IVOA Interop, Victoria, 17/05/2006, page 4 How this all is organized? Firewall Migration is forcing us to divide the Local Grid To be installed 10 nodes, RHEL dual cpu, 6GB RAM Scigrid linked to ESTEC GRID ESRIN GRID UCM GRID

Science Operations & Data Systems Division Research & Scientific Support Department Christophe ARVISET Science Operations & Data Systems Division Research & Scientific Support Department IVOA Interop, Victoria, 17/05/2006, page 5 GRID for the day to day analysis The user could use the local Grid to: Run IDL Run SAS, OSA or whatever mission-specific software interactively Data visualization Submit massive data processing batch jobs, … The user will use his/her desktop for: Internet browsing Writing documents MP3 Data visualization SAS OSA iPython Heasoft

Science Operations & Data Systems Division Research & Scientific Support Department Christophe ARVISET Science Operations & Data Systems Division Research & Scientific Support Department IVOA Interop, Victoria, 17/05/2006, page 6 GRID for Project Data Processing Standard Pipeline Data Processing Bulk and On-the-Fly (OTF) Processing of data Run the Pipeline on many observations Automatic process, no human intervention Perfectly tuned for GRID processing Migrate XMM-Newton Archive OTF to scigrid (2006) Run Herschel Pipeline Data Processing on scigrid (2007) Increase number of nodes GAIA data processing in Several hundreds of machines ?

Science Operations & Data Systems Division Research & Scientific Support Department Christophe ARVISET Science Operations & Data Systems Division Research & Scientific Support Department IVOA Interop, Victoria, 17/05/2006, page 7 GRID and the Virtual Observatory Interaction ESA VO and ESAC GRID teams being built up Main long term goals ESA VO as a data provider ESA VO as a data processing centre Use the GRID with VO protocols DAL, SSO/Certificates, SkyNode/Xmatch, … Build a VOSpace located on the ESA VO Grid Like to start linking ESAVO GRID with other VO GRIDs

Science Operations & Data Systems Division Research & Scientific Support Department Christophe ARVISET Science Operations & Data Systems Division Research & Scientific Support Department IVOA Interop, Victoria, 17/05/2006, page 8 Example : Xmatch on GRID (TBD)