1. Dominik Stoklosa Poznan Supercomputing and Networking Center, Supercomputing Department EGEE 2007 Budapest, Hungary, October 1-5 Workflow management in Remote Instrumentation Infrastructures – e-VLBI experiences

2. Introduction to the e-VLBI Introduction to the e-VLBI EXPReS project EXPReS project PSNC in EXPReS - FABRIC PSNC in EXPReS - FABRIC System design System design Managing data flows Managing data flows Outline

3. Introduction to the e-VLBI VLBI is a technique, in which physically independent and widely separated radio telescopes observe the same region of sky simultaneously, in order to generate very high-resolution continuum and spectral-line images of cosmic radio sources Telescopes are usually separated by thousands of kilometres Data from each telescope are digitally sampled and stored locally, using high-capacity magnetic tape systems and magnetic disk-array systems Data are sent and correlated at the central point (JIVE) The total flow of data into the central processor is approximately 10- 100 Terabytes per single observation, after processing this is reduced to 10-100 Gbytes.

4. Radio Telescopes Arecibo, Chile Onsala, Sweden RT4, Poland

5. Westerbork / Very Large Array

6. 408 Mhz optical 1.4 Ghz Radio / Optical

7. Introduction to EXPReS EXPReS – the objective is to create a production-level electronic VLBI (e-VLBI) service, in which the radio telescopes are reliably connected to the central data processor at JIVE via a high-speed optical-fibre communication network. Project Details Three years, started March 2006 International collaboration Funded at 3.9 million EUR FP6, Contract #026642

8. Introduction EXPReS partners 19 partners, 21 telescopes, 6 continents

9. PSNC in EXPReS EXPReS – a Real-time e-VLBI Radio Telescope - JRA1: Future Arrays of Broadband Radio-Telescopes on Internet Computing (FABRIC) - Grid – VLBI collaboration - Grid Workflow management - Grid Routing Creating solution for incorporating Grid resources for distributed correlation using existing infrastructure.

10. Once upon a time (1) Everything was slow Telescopes collected data on tapes Sent via postal mail Hard drive arrays slightly improved the situation The entire cycle could easily require 6 months or more

11. Once upon a time (2) Hardware correlator; the EVN MkIV data correlator at JIVE dedicated, purpose designed/built hardware a super computer; ~100 T ops/sec

12. Today / In the near future Data can be transferred over the network Each stage of the process can be speeded up GRID resources Software correlator e-VLBI - electronic VLBI

13. System design – data flows (1)

14. System design – data flows (2)

15. WFM – phase 1 Definition of radio telescopes – automatically based on the observation schedule

16. WFM – phase 2 Definition of file servers (each file sever is responsible for capturing data from RT)

17. WFM – phase 3 Definition of correlation nodes and data flows between components

18. WFM – properties Definition of resource properties

19. Remote Instrumentation in Next Gen Grids www.ringrid.eu RINGrid – SSA project funded under 6 th FP Partners from UK, Austria, Greece, Italy, Romania, Bulgaria, Mexico and Brazil RING rid objectives: Identification of instruments and user communities, definition of requirements Trends definition and recommendations for designing next- generation Remote Instrumentation Services Promoting equal access to European e-Infrastructure opportunities http://www.ringrid.eu/

20. Thank you for your attention http://www.expres-eu.org/ d.stoklosa@man.poznan.pl EXPReS is made possible through the support of the EC, 6 th FP, Contract #026642