Dominik Stoklosa Poznan Supercomputing and Networking Center, Supercomputing Department EGEE 2007 Budapest, Hungary, October 1-5 Workflow management in Remote Instrumentation Infrastructures – e-VLBI experiences

Introduction to the e-VLBI Introduction to the e-VLBI EXPReS project EXPReS project PSNC in EXPReS - FABRIC PSNC in EXPReS - FABRIC System design System design Managing data flows Managing data flows Outline

Introduction to the e-VLBI VLBI is a technique, in which physically independent and widely separated radio telescopes observe the same region of sky simultaneously, in order to generate very high-resolution continuum and spectral-line images of cosmic radio sources Telescopes are usually separated by thousands of kilometres Data from each telescope are digitally sampled and stored locally, using high-capacity magnetic tape systems and magnetic disk-array systems Data are sent and correlated at the central point (JIVE) The total flow of data into the central processor is approximately Terabytes per single observation, after processing this is reduced to Gbytes.

Radio Telescopes Arecibo, Chile Onsala, Sweden RT4, Poland

Westerbork / Very Large Array

408 Mhz optical 1.4 Ghz Radio / Optical

Introduction to EXPReS EXPReS – the objective is to create a production-level electronic VLBI (e-VLBI) service, in which the radio telescopes are reliably connected to the central data processor at JIVE via a high-speed optical-fibre communication network. Project Details Three years, started March 2006 International collaboration Funded at 3.9 million EUR FP6, Contract #026642

Introduction EXPReS partners 19 partners, 21 telescopes, 6 continents

PSNC in EXPReS EXPReS – a Real-time e-VLBI Radio Telescope - JRA1: Future Arrays of Broadband Radio-Telescopes on Internet Computing (FABRIC) - Grid – VLBI collaboration - Grid Workflow management - Grid Routing Creating solution for incorporating Grid resources for distributed correlation using existing infrastructure.

Once upon a time (1) Everything was slow Telescopes collected data on tapes Sent via postal mail Hard drive arrays slightly improved the situation The entire cycle could easily require 6 months or more

Once upon a time (2) Hardware correlator; the EVN MkIV data correlator at JIVE dedicated, purpose designed/built hardware a super computer; ~100 T ops/sec

Today / In the near future Data can be transferred over the network Each stage of the process can be speeded up GRID resources Software correlator e-VLBI - electronic VLBI

System design – data flows (1)

System design – data flows (2)

WFM – phase 1 Definition of radio telescopes – automatically based on the observation schedule

WFM – phase 2 Definition of file servers (each file sever is responsible for capturing data from RT)

WFM – phase 3 Definition of correlation nodes and data flows between components

WFM – properties Definition of resource properties

Remote Instrumentation in Next Gen Grids RINGrid – SSA project funded under 6 th FP Partners from UK, Austria, Greece, Italy, Romania, Bulgaria, Mexico and Brazil RING rid objectives: Identification of instruments and user communities, definition of requirements Trends definition and recommendations for designing next- generation Remote Instrumentation Services Promoting equal access to European e-Infrastructure opportunities

Thank you for your attention EXPReS is made possible through the support of the EC, 6 th FP, Contract #026642