1. AMPATH, FIU, Miami, 31 January 2003 eVLBI Research in the European VLBI Network Steve Parsley Joint Institute for VLBI in Europe Dwingeloo, Netherlands parsley@jiv.nl

2. AMPATH, FIU, Miami, 31 January 2003 Outline European VLBI Network (EVN) European Research Networks Fibre Connection to Dwingeloo iGrid Demonstration EVN-NREN: Proof of Concept Project

3. AMPATH, FIU, Miami, 31 January 2003

4. The European VLBI Network

5. EVN Telescopes

6. AMPATH, FIU, Miami, 31 January 2003 Correlator

8. AMPATH, FIU, Miami, 31 January 2003

9. European Research Networks National NREN = National Research and Education Network UKERNA, SURFnet, GARR, DFN, PSNC, NORDUnet ….. International GÉANT = Pan-European Research Network

10. 0 SISI GRGR CZCZ IEIE LULU EEEE SKSK ILIL LTLTCYCY ESES HUHU ITIT FRFR UKUK BEBE NLNL DEDE SESE CHCH PTPT ATAT GEANT GEANT 2.5 G 1.2 G 622M 155 M BGBG LVLV RORO HRHR PLPL 45 M 34 M 310 M GÉANT: Access of NRENs to GÉANT EVN telescope

11. AMPATH, FIU, Miami, 31 January 2003 0UKUK FRFR CHCH SESE ITITRORO HRHR EEEELVLVLTLT IEIE NLNL BEBE DEDE PLPL CZCZ HUHUATAT SISI SKSK LULU ESES PTPT GRGR BGBG CYCYILIL 622 34 34 45 622 622 155 155 155 155 34 34 155 622 155 34 45 10 Gbps 10 Gbps 2.5 Gbps 2.5 Gbps SE - PoP for Nordunet GÉANT: The connectivity at 10 Gbps

12. 3 colours @1Gbps from Dwingeloo to Amsterdam Internet Exchange 20 km of new fibre 180km to SARA via installed link Dwingeloo

13. AMPATH, FIU, Miami, 31 January 2003 Network SARA – ASTRON / JIVE GXSARAASTRON / JIVE Access to NetherLight Access to SURFnet5 (ar5.amsterdam1) GbE Project eVLBI GbE ISDN Cisco2620SNET-LAN 4912G Cisco1721 Cisco15216 EDFA (2x) Cisco15252 DWDM Cisco15252 DWDM RS 232 GbE ODF Amsterdam1Emmeloord1Dwingeloo1 GbE ASTRON router

14. AMPATH, FIU, Miami, 31 January 2003 iGrid Demonstration 24-26 September 2002 Amsterdam Science and Technology Centre (WTCW) The Netherlands Grid 2 oo 2 www.startap.net/igrid2002 www.igrid2002.org (COMING SOON) The International Virtual Laboratory Call for Applications with Insatiable Bandwidth Appetites! “We hereby challenge the international research community to demonstrate applications that benefit from huge amounts of bandwidth!” i

15. AMPATH, FIU, Miami, 31 January 2003 Westerbork Jodrell Bank SARA Dwingeloo Shared IP over GÉANT GEthernet over SURFnet Lambda Future fibre Link (now it’s a truck) Live telescope pictures Image Fringe Display JIVE Correlator About VLBI Other VLBI collaborators Science World of VLBI VLBI Technique & Operations vlb

16. AMPATH, FIU, Miami, 31 January 2003 vlbiGrid Objectives Trial use of UDP for VLBI data transfer across production research networks: Data rates Packet loss Packet reordering Exercise new link to JIVE Demonstrate eVLBI fringes Publicise VLBI to network researchers

17. AMPATH, FIU, Miami, 31 January 2003 iGrid Network Routing Geant Core SuperJANET4 Core Network NNW MAN Manchester Server DWDM link SURFnet Core iGrid2002 Amsterdam 1 Gbit Eth. 2.5 Gbit Pos 10 Gbit PCEVN

18. Data Transfer Process: Raid0 Disc UDP Data Raid0 Disc RingBuffer TCP Control Web Interface

19. Receiving Packets Time 1234612346 Incoming packet ids 1 2 3 4 Move 6 Continue with 7 Packets have Header with packet id + Data Put header directly into control area Put data directly into ring buffer No extra copy Assume UDP packets arrive in order none lost Deposit data into next consecutive slot Inspect Header Move data onward to correct location Record: The received inter-packet spacing 1-way delay

20. AMPATH, FIU, Miami, 31 January 2003 iGrid Results UDP across GÉANT (server-server) : ~500Mbit/s <6E-5 lost packets <1E-5 reordered packets (all exchanges) FTP to Dwingeloo: ~40Mbit/s Normal fringes detected PR: Great success, new link used three days after first light Much interest in VLBI data rates and volumes

22. AMPATH, FIU, Miami, 31 January 2003 iGrid Lessons Learned 500Mbit/s VLBI data transfer on the production network using a simple UDP based protocol is feasible. Acceptable packet loss is achievable. Packets may be re-ordered. A simple data moving strategy corrects for this. Control/monitoring data should be on a separate, reliable channel. End hosts must have sufficient power in both compute cycles and input/out capability. An initial specification would include: 64bit 66MHz PCI bus 2 GHz processor Gigabit Ethernet interface: Intel Pro 1000 SysKonnect Fast disks with high speed buses Maxtor D740 EIDE 133

23. AMPATH, FIU, Miami, 31 January 2003 Proof of Concept Project (PoC) Target: Up to 5 radio telescope sites (not incl NL) connected in real-time to JIVE correlator Duration: Up to end of GÉANT contract (Nov 2004) Support: Best effort IP service transiting NRENs and GÉANT No significant upgrades to GÉANT (initially) Support 512M and 1G real time modes of operation Use existing NREN access ports Limited resilience

24. AMPATH, FIU, Miami, 31 January 2003 Proof of Concept Participants DANTE Pan-European Network GARR Italian NREN UKERNAUK NREN PSNC Polish NREN DFN German NREN SURFnetDutch NREN KTHNOC/NORDUnet Nordic NREN Manchester UniversityNetwork application software JIVEEVN Correlator Westerbork telescopeNetherlands Onsala Space Observatory Sweden MPIfRGermany Jodrell Bank UK TCfAPoland CNR IRA Italy

25. AMPATH, FIU, Miami, 31 January 2003 Success Criteria (for PoC) NREN/GÉANT point of view: Connect at least three sites to JIVE Observe significant usage (time and BW) EVN point of view: Same-day imaging of 12hour, 1Gb/s observation Real time network verification of transient phenomenon

26. AMPATH, FIU, Miami, 31 January 2003 Traffic Engineering on GÉANT...

27. GÉANT Nodes Affected by EVN Traffic UK SE FR NL BE DE1 DE2 CZ PL CH IT AT SURFnetJANETGARRPSNCDFN NORDUnet 2.5G 10G

28. Failure Scenarios #1 UK SE FR NL BE DE1 DE2 CZ PL CH IT AT SURFnetJANETGARRPSNCDFN NORDUnet 2.5G 10G

29. Failure Scenarios #1 UK SE FR NL BE DE1 DE2 CZ PL CH IT AT SURFnetJANETGARRPSNCDFN NORDUnet 2.5G 10G

30. Failure Scenarios #2 UK SE FR NL BE DE1 DE2 CZ PL CH IT AT SURFnetJANETGARRPSNCDFN NORDUnet 2.5G 10G

31. Failure Scenarios #2 UK SE FR NL BE DE1 DE2 CZ PL CH IT AT SURFnetJANETGARRPSNCDFN NORDUnet 2.5G 10G

32. Potential Upgrades to GÉANT UK SE FR NL BE DE1 DE2 CZ PL CH IT AT SURFnetJANETGARRPSNCDFN NORDUnet 2.5G 10G

33. Failure Scenarios #3 UK SE FR NL BE DE1 DE2 CZ PL CH IT AT SURFnetJANETGARRPSNCDFN NORDUnet 2.5G 10G

34. Failure Scenarios #3 UK SE FR NL BE DE1 DE2 CZ PL CH IT AT SURFnetJANETGARRPSNCDFN NORDUnet 2.5G 10G

35. Failure Scenarios #4 UK SE FR NL BE DE1 DE2 CZ PL CH IT AT SURFnetJANETGARRPSNCDFN NORDUnet 2.5G 10G

36. Failure Scenarios #4 UK SE FR NL BE DE1 DE2 CZ PL CH IT AT SURFnetJANETGARRPSNCDFN NORDUnet 2.5G 10G

37. AMPATH, FIU, Miami, 31 January 2003 Some Technical Issues Packet re-ordering due to architecture of Juniper M160 routers in GÉANT backbone. VLBI UDP application must fix this. VLBI data rates are restricted to “octaves” : - 256M, 512M, 1G “1Gbps” eVLBI flows are >1Gbps  two GE channels will be needed to support "1Gbps" from each telescope.

38. AMPATH, FIU, Miami, 31 January 2003 SURFnet GÉANT NREN1 NREN2 NREN3 IP1IP2IP3IP4 JIVE

39. AMPATH, FIU, Miami, 31 January 2003 PoC Design NREN SURFnet GE JIVE Corr. Separate, GE point-to-point connection for each telescope GÉANT GE link on /30 subnet between VLAN interface on the SURFnet access router and the GE NIC at Dwingeloo. Add second GE NIC for “1Gb/s” each carrying 512Mb/s

40. AMPATH, FIU, Miami, 31 January 2003 Local Loops - Status Italy: GARR : Backbone capacity - 2.5Gbps. Medicina radio telescope: 30km from Bologna POP. Good connectivity (dark fibre) expected by end of 2003. (3-4km dig to radio telescope.) Poland: PSNC: 10GE link from Torun to Poznan (location of GÉANT POP) ready in June 2003. Torun radio telescope: 15km, Torun to telescope in place. 100m and terminal equipment within 3 months. Germany: DFN: No backbone problems expected. Bonn ring on-line April’03. 5-7km connection to DFN POP needed but not planned. Effelsberg Radio Telescope: Fibre to telescope in place bar 5km copper section. UK: UKERNA: no problems to support the eVLBI traffic on the SuperJANET backbone. 2.5G backup access port to GÉANT planned. Jodrell Bank: significant problem with local loop to JB (BT quoted £1M for the first year) Options being explored include “piggy-back” on eMERLIN, 30Gbit/s infrastructure ~June 2003 Sweden: NORDUnet: two, redundant 2.5Gbit/s links to GEANT in Stockholm available. SUNET: 10Gbit/s backbone with 2X2.5Gbit/sSRP links between Chalmer University and Stockholm. Onsala Space Observatory: 40km of fibre required. Installation costs of up to €150k have been quoted with €20k+ annual charges. The Netherlands: SURFnet: 2.5Gbit/s border connection will soon be upgraded to 10Gbit/s JIVE: 190km dark fibre link from JIVE to Amsterdam (SARA) in place. Westerbork: 24 fibre link directly to Dwingeloo by June‘03

41. AMPATH, FIU, Miami, 31 January 2003 Project Timeline

42. AMPATH, FIU, Miami, 31 January 2003 SURFnet GÉANT NREN1 NREN2 NREN3 IP1IP2IP3IP4 JIVE

43. AMPATH, FIU, Miami, 31 January 2003 UKLight Proposal Initiative to attract funding to create: UK point-of-access (PoA) to the international test-bed Intra UK optical test-bed Draft projects: Development of a common optical control plane Secure Optical Community Services A project to use a dark fibre infrastructure An applications demonstration based upon very long baseline interferometry (VLBI) for Radio Astronomy

44. AMPATH, FIU, Miami, 31 January 2003 Dedicated Lambda VLBI Network

45. AMPATH, FIU, Miami, 31 January 2003 2 nd eVLBI Workshop 15-16 May 2003, Dwingeloo, Netherlands First Announcement10 th Jan ‘03 Second Announcement1 st March ‘03 Abstract Deadline19 th March ‘03 Final Announcement3 rd April ‘03 Registration Deadline20 th April ‘03 Hotel Deadline15 th April ‘03 Workshop15-16 th May ’03 http://www.jive.nl/jive/evlbi_ws/meeting