HAH 2007/05/09 Connecting LOFAR to the GRID Hanno Holties, ASTRON IVOA - OGF Astro-RG Work Shop Manchester, 9th May 2007.

Slides:

Advertisements

Similar presentations

Dominik Stoklosa Poznan Supercomputing and Networking Center, Supercomputing Department EGEE 2007 Budapest, Hungary, October 1-5 Workflow management in.

Advertisements

What does LOFAR have to do with the Virtual Observatory (VO)? LOFAR Science Day 16 December 2003 Melbourne David Barnes The University of Melbourne.

The Australian Virtual Observatory e-Science Meeting School of Physics, March 2003 David Barnes.

Slide: 1 Welcome to the workshop ESRFUP-WP7 User Single Entry Point.

GRID Activities at ESAC Science Archives and Computer Engineering Unit Science Operations Department ESA/ESAC – Madrid, Spain.

Lofar Survey workshop 8/3/2007 Lofar -Surveys and AstroWise OmegaCEN NOVA – Kapteyn Institute -RuG 8 March 2007 Lofar Survey Workshop OmegaCEN NOVA – Kapteyn.

XLDB -Lyon Aug 2009 Astro-WISE -> TarGet OmegaCEN – TarGet Kapteyn Astronomical Institute University of Groningen 28 August 2009 UU OmegaCEN – TarGet Kapteyn.

HPC-Europa2 Funding research visits in Europe

LIGO- GXXXXXX-XX-X Advanced LIGO Data & Computing Material for the breakout session NSF review of the Advanced LIGO proposal Albert Lazzarini Caltech,

Astronomical GRID Applications at ESAC Science Archives and Computer Engineering Unit Science Operations Department ESA/ESAC.

Assessment of RFI measurements for LOFAR Mark Bentum, Albert-Jan Boonstra, Rob Millenaar ASTRON, The Netherlands Telecommunication Engineering, University.

05/24/2004The 12th UN/ESA Workshop on Basic Space Science Chinese Virtual Observatory (China-VO) Chenzhou Cui Chinese Virtual Observatory Project National.

Slide 1 Experiences with NMI R2 Grids Software at Michigan Shawn McKee April 8, 2003 Internet2 Spring Meeting.

E-VLBI at ≥ 1 Gbps -- “unlimited” networks? Tasso Tzioumis Australia Telescope National Facility (ATNF) 4 November 2008.

LOFAR AND AFRICA Daan du Toit DST – South Africa.

09-Sept-2004 NeSC UKLight Town Meeting Peter Clarke, UKLight Town Meeting Welcome, background and & programme for the day Peter Clarke.

A short introduction to the Worldwide LHC Computing Grid Maarten Litmaath (CERN)

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Related Projects Dieter Kranzlmüller Deputy.

Results of the HPC in Europe Taskforce (HET) e-IRG Workshop Kimmo Koski CSC – The Finnish IT Center for Science April 19 th, 2007.

Astro-WISE & Grid Fokke Dijkstra – Donald Smits Centre for Information Technology Andrey Belikov – OmegaCEN, Kapteyn institute University of Groningen.

Netherlands Institute for Radio Astronomy 1 ASTRON is part of the Netherlands Organisation for Scientific Research (NWO) LOFAR Operations and Schedule.

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks LOFAR Archive Information System Kor Begeman.

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks ?? Athens, May 5-6th 2009 Community Support.

14 Aug 08DOE Review John Huth ATLAS Computing at Harvard John Huth.

Grid Computing Status Report Jeff Templon PDP Group, NIKHEF NIKHEF Scientific Advisory Committee 20 May 2005.

SouthGrid SouthGrid SouthGrid is a distributed Tier 2 centre, one of four setup in the UK as part of the GridPP project. SouthGrid.

Federated Discovery and Access in Astronomy Robert Hanisch (NIST), Ray Plante (NCSA)

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Integration of Astro-WISE with Grid storage.

Research Networks and Astronomy Richard Schilizzi Joint Institute for VLBI in Europe

EScience May 2007 From Photons to Petabytes: Astronomy in the Era of Large Scale Surveys and Virtual Observatories R. Chris Smith NOAO/CTIO, LSST.

The DutchGrid Platform – An Overview – 1 DutchGrid today and tomorrow David Groep, NIKHEF The DutchGrid Platform Large-scale Distributed Computing.

EGEE is a project funded by the European Union under contract IST HEP Use Cases for Grid Computing J. A. Templon Undecided (NIKHEF) Grid Tutorial,

The GRelC Project: architecture, history and a use case in the environmental domain G. Aloisio - S. Fiore The Climate-G testbed is an interdisciplinary.

EVLA Software Bryan Butler. 2007May22EVLA SAGE Meeting2 Requirements and Goals of EVLA Software Maximize scientific throughput of the instrument At a.

Key prototype applications Grid Computing Grid computing is increasingly perceived as the main enabling technology for facilitating multi-institutional.

TOPS “Technology for Optical Pixel Streaming” Paul Wielinga Division Manager High Performance Networking SARA Computing and Networking Services

Prologue Major VO objective: “Data at your fingertips” (Lawrence) VO-user’s desire: “going backstage” –Data quality assessment by user (talk Gilmore)

ESFRI & e-Infrastructure Collaborations, EGEE’09 Krzysztof Wrona September 21 st, 2009 European XFEL.

Data Archives: Migration and Maintenance Douglas J. Mink Telescope Data Center Smithsonian Astrophysical Observatory NSF

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks David Fergusson, Emidio Giorgio, Gergely.

ATLAS WAN Requirements at BNL Slides Extracted From Presentation Given By Bruce G. Gibbard 13 December 2004.

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarksEGEE-III INFSO-RI Astro-Wise and EGEE.

EURO-VO: GRID and VO Lofar Information System Design OmegaCEN Kapteyn Institute TARGET- Computing Center University Groningen Garching, 10 April 2008 Lofar.

Lofar Information System on GRID A.N.Belikov. Lofar Long Term Archive Prototypes: EGEE Astro-WISE Requirements to data storage Tiers Astro-WISE adaptation.

Introduction to the VO ESAVO ESA/ESAC – Madrid, Spain.

NeSC Training Team Enabling, facilitating and delivering quality training in the UK and Internationally.

1 e-Arts and Humanities Scoping an e-Science Agenda Sheila Anderson Arts and Humanities Data Service Arts and Humanities e-Science Support Centre King’s.

IAG – Israel Academic Grid, EGEE and HEP in Israel Prof. David Horn Tel Aviv University.

Netherlands Institute for Radio Astronomy Big Data Radio Astronomy A VRC for SKA and its pathfinders Hanno Holties March 28 th 2012.

1 eScience in Astronomy: Grid & VO GAVO III KickOff eScience in Astronomy: VO & GRID eScience: making the most advanced tools of IT available to scientists.

A Computing Tier 2 Node Eric Fede – LAPP/IN2P3. 2 Eric Fede – 1st Chinese-French Workshop Plan What is a Tier 2 –Context and definition To be a Tier 2.

The status of IHEP Beijing Site WLCG Asia-Pacific Workshop Yaodong CHENG IHEP, China 01 December 2006.

European Grid Initiative EGI FedCloud User Support coordination meeting Diego Scardaci - EGI.eu/INFN 01/04/2015.

DATA ACCESS and DATA MANAGEMENT CHALLENGES in CMS.

Virtual Laboratory Amsterdam L.O. (Bob) Hertzberger Computer Architecture and Parallel Systems Group Department of Computer Science Universiteit van Amsterdam.

Open Science cloud access to LOFAR data and compute

Accessing the VI-SEEM infrastructure

Pasquale Migliozzi INFN Napoli

Applications Using the EGEE Grid Infrastructure

Taking a researcher’s e-Infrastructure to the next level

Hannelore Hammerle NA2 F2F meeting, Manchester 8 May 2007

EGEE NA4 Lofar Lofar Information System Design OmegaCEN

Статус ГРИД-кластера ИЯФ СО РАН.

ROSCOE Robust Scientific Communities for EGI

LOFAR Beam Formed (BF) Data & Pipeline Overview September 9, 2010

TOPS “Technology for Optical Pixel Streaming”

Observing with Modern Observatories (the data flow)

TOPS “Technology for Optical Pixel Streaming”

Princeton Center for Complex Materials: DMR

Maria Teresa Capria December 15, 2009 Paris – VOPlaneto 2009

Presentation transcript:

HAH 2007/05/09 Connecting LOFAR to the GRID Hanno Holties, ASTRON IVOA - OGF Astro-RG Work Shop Manchester, 9th May 2007

HAH 2007/05/09

First result: 96 LBA antennas Upto 400 m apart Dynamic range: ~1000:1 Cas A (strongest source) removed from image

HAH 2007/05/09 LOFAR operational model

HAH 2007/05/09 LOFAR Central systems Process ‘raw’ data Calibration Reduce in size Temporary Storage Raw data: 5 days Processed dataproducts: 1 month Push data to users Science centers subscribe & publish storage location LOFAR scheduler initiates …or interactive

HAH 2007/05/09 The LOFAR Network & GRID 700 Gbps dedicated fibre 2 Gbps lightpaths 20 Gbps lightpaths GRID Computing centers Science centers LOFAR instrument (Dutch part) International LOFAR stations Science center with local storage

HAH 2007/05/09 GRID utilization Long term storage In general at computing center Searchable database Data mining Virtual Observatory Offline processing Reprocessing Source extraction Differencing (search for variable or new sources)

HAH 2007/05/09 Storage requirements Required storage capacity depends on Datatype (timeseries, visibilities, images) Number of stations involved Required time, frequency, and spatial resolution Prognosis 1.8 PB/Yr

HAH 2007/05/09 Processing requirements In process of being defined Applications still being developed GRID awareness of scientists just starting Proximity to, and available bandwidth for, storage IO limited Operate on Multi-Terabyte datasets Mostly embarrassingly parallel Environment Application requirements HPC tuning requires system details Shield from scientist

HAH 2007/05/09 Where are we? Working group LOFAR & GRID Meets on monthly basis Users, RCG, SARA, NIKHEF, ASTRON/LOFAR EGEE Tutorial before the summer LOFAR VO (in GRID context) Group for each KSP (to become per science center) Resources at SARA; RCG to follow shortly Glite TAR-UI installed on storage frontend Some groups have started to get their feet wet Kapteyn Institute UvA Transients group

HAH 2007/05/09 eLOFAR: The extended LOFAR Interest from international partners in: Germany ( UK ( France Sweden Italy Poland Aiming to extend LOFAR with: International stations Science Centers. Computing centers

HAH 2007/05/09 LOFAR challenges for GRID Storage 20 Gbps sustainable bandwidth Multi-Petabyte growth per year Interactive databases Processing Operate on Multi-TB datasets Applications with varying requirements Usability Reliable facilities Technical details shielded from (general) user

HAH 2007/05/09