1. EURO-VO: GRID and VO Lofar Information System Design OmegaCEN Kapteyn Institute TARGET- Computing Center University Groningen Garching, 10 April 2008 Lofar Information System Design OmegaCEN Kapteyn Institute TARGET- Computing Center University Groningen Garching, 10 April 2008 Edwin A. Valentijn EURO-VO DCA GRID and the VO

2. EURO-VO: GRID and VO LOFAR @ (20-)30-80 MHz and 120-240 MHz -EoR - transients -Survey -astroparticles -EoR - transients -Survey -astroparticles

3. EURO-VO: GRID and VO

4. E-Lofar: from experiment to observatory

5. EURO-VO: GRID and VO LOFAR in short Large distributed radio telescope –In NL 2008: initial roll out 20 stations –In NL 2009: 18 central (24 Gbps) + 18 remote (2 Gbps) phased array antenna stations each ~4 soccerfields size –Full scale aperture synthesis array, extends 100 km Two main bands –High Band ~ 7,500 tiles 120 – 240 MHz –Low Band ~ 7,500 dipoles 20 – 80 MHz Digital Radio 40 MHz processing bandwidth –Extreme tunability in time/space/frequency –Large instantaneous sky coverage Fibre network, Software Correlator Run as a broad common-user observatory –Making the transition now from “project” to “observatory” Large distributed radio telescope –In NL 2008: initial roll out 20 stations –In NL 2009: 18 central (24 Gbps) + 18 remote (2 Gbps) phased array antenna stations each ~4 soccerfields size –Full scale aperture synthesis array, extends 100 km Two main bands –High Band ~ 7,500 tiles 120 – 240 MHz –Low Band ~ 7,500 dipoles 20 – 80 MHz Digital Radio 40 MHz processing bandwidth –Extreme tunability in time/space/frequency –Large instantaneous sky coverage Fibre network, Software Correlator Run as a broad common-user observatory –Making the transition now from “project” to “observatory”

6. E-LOFAR Participants GLOW: -- Effelsberg station delivered -- Garching, Potsdam, Tautenburg stations ordered --Jülich station federal funding proposal submitted LOFAR-UK: – Chilbolton, Cambridge, Jodrell, Edinburgh – 1 station funded, site to be chosen --3 more stations Sweden: – Onsala station funded + Poland, Italy, Austria, Bulgaria, Ireland, France GLOW: -- Effelsberg station delivered -- Garching, Potsdam, Tautenburg stations ordered --Jülich station federal funding proposal submitted LOFAR-UK: – Chilbolton, Cambridge, Jodrell, Edinburgh – 1 station funded, site to be chosen --3 more stations Sweden: – Onsala station funded + Poland, Italy, Austria, Bulgaria, Ireland, France EURO-VO: GRID and VO

7. Wires in the field EURO-VO: GRID and VO

8. Station based processing –Receiver –Filter –Beamformer –Buffer –Receiver –Filter –Beamformer –Buffer –Receiver –Filter –Beamformer –Buffer –Receiver –Filter –Beamformer –Buffer –10 GbE –1 GbE –Used for 1 beam & station cross correlation Input data rate: ~ 460 Gbps Output data rate: ~ 2 Gbps Processing capacity: ~ 1.5 Tmul/s Storage capacity: 96 Gbyte

9. Lofar: Data Stream EURO-VO: GRID and VO

10. Lofar Storage

11. Lofar Long term (>2weeks) archive and user enabling ADD EURO-VO: GRID and VO

12. STREAM of MetaData Design: data driven vs. compute driven from the field to central processor Distributed – p2p / federated –Communities –Dataservers – own + EGEE GRID –Compute servers – own+ EGEE GRID AstroWISE EURO - VO Design: data driven vs. compute driven from the field to central processor Distributed – p2p / federated –Communities –Dataservers – own + EGEE GRID –Compute servers – own+ EGEE GRID AstroWISE EURO - VO EURO-VO: GRID and VO

13. LoWISE design

14. Lofar-WISE: Data Storage –Groningen –SARA Amsterdam EURO-VO: GRID and VO

15. IBM – GPFS filesystem Connect to AstroWISE –Storage table IBM – GPFS filesystem Connect to AstroWISE –Storage table EURO-VO: GRID and VO

16. LoWise- concepts All Tier 1s on an equal footing Centrally operated GPFS distributed filesystem File retrieval –Db viewer (web) –Prompt (db query) –Pathname – data server –gLite – Grid ftp Single distributed db- with all Meta and source data –Valid --Quality -- ready for publication flags Visibility of data –Owner –Project –Archive --EURO-VO Authentication / Assigned resources –AstroWISE Oracle system All Tier 1s on an equal footing Centrally operated GPFS distributed filesystem File retrieval –Db viewer (web) –Prompt (db query) –Pathname – data server –gLite – Grid ftp Single distributed db- with all Meta and source data –Valid --Quality -- ready for publication flags Visibility of data –Owner –Project –Archive --EURO-VO Authentication / Assigned resources –AstroWISE Oracle system EURO-VO: GRID and VO

17. Processing - options Tier 0 – CP Tier 1 – re-processing –Local clusters –EGEE GRID Tier 2 –On Tier 1 –Local anarchy Tier 0 – CP Tier 1 – re-processing –Local clusters –EGEE GRID Tier 2 –On Tier 1 –Local anarchy EURO-VO: GRID and VO

18. AstroWISE-Up

19. EURO-VO: GRID and VO –VST - Virtual Survey Telescope

20. EURO-VO: GRID and VO –www.astro-wise.org

21. EURO-VO: GRID and VO

22. Astro-WISE: Services EURO-VO: GRID and VO

23. TARGET diagram EURO-VO: GRID and VO

24. QC - calibration scientist monitoring

25. EURO-VO: GRID and VO Extreme data lineage

26. Astro-WISE: Grid Computing –selection of Target in Astro-Wise –storage element defined a priori –EGEE computing EURO-VO: GRID and VO

28. LoWise and Euro-VO EURO-VO: GRID and VO

29. GRID- EuroVO –Awe visualization

30. EURO-VO: GRID and VO –AI – Cultural Heritage

31. EURO-VO: GRID and VO http://omegate.astro.rug.nl:9001 /Monk