4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE e-VLBI and EXPReS Arpad Szomoru

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE Outline Current status, recent results Network test e-VLBI and the EXPReS proposal Conclusions, future

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE

Why e-VLBI ? Reliability – real-time feedback to the telescopes Sensitivity – sustained high data rates possible Logistics – No media management Rapid science results: Geodesy Precision spacecraft navigation

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE e-VLBI Proof-of-Concept Project DANTE/GÉANT Pan-European Network SURFnetDutch NREN GARR Italian NREN UKERNAUK NREN PSNC Polish NREN DFN German NREN KTHNOC/NORDUnet Nordic NREN Manchester UniversityNetwork application software JIVEEVN Correlator Westerbork telescopeNetherlands Onsala Space Observatory Sweden MROFinland MPIfRGermany Jodrell Bank UK TCfAPoland CNR IRA Italy

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE 0 SISI GRGR CZCZ IEIE LULU EEEE SKSK ILIL LTLTCYCY ESES HUHU ITIT FRFR UKUK BEBE NLNL DEDE SESE CHCH PTPT ATAT GEANT GEANT 2.5 G 1.2 G 622M 155 M BGBG LVLV RORO HRHR PLPL 45 M 34 M 310 M GÉANT: Access of NRENs to GÉANT EVN telescope

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE for the EVN and JIVE Standards:- Protocols, parameter tuning, procedures at telescope and correlator. New Capabilities:- Higher data rates, improved reliability, quicker response. for GÉANT and the NRENs To see significant network usage with multiple Gbit streams converging on JIVE. Minimum is three telescopes (not including Westerbork) Must be seen to enable new science and not just solve an existing data-transport problem Real-time operation is seen as the ultimate aim, buffered operation accepted as a development stage. POC targets:

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE

Fibre pair from Amsterdam Optical Splitters GE lines to correlator LX Optics converters Cisco ONS Correlator Interface Current status: JIVE: 6 lambdas via Netherlight, each capable of 1 Gbps 1 Gbps connections to Westerbork, Torun, Onsala, Jodrell 155 Mbps to Arecibo, 128 to Cambridge Medicina being connected at 1 Gbps now, Metsahovi 2006 Current status: JIVE: 6 lambdas via Netherlight, each capable of 1 Gbps 1 Gbps connections to Westerbork, Torun, Onsala, Jodrell 155 Mbps to Arecibo, 128 to Cambridge Medicina being connected at 1 Gbps now, Metsahovi 2006

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE Network testing Use existing protocols on currently available hardware TCP maximal reliability Not really required Sensitive to congestion Lot of fine-tuning necessary And possible UDP connectionless Unaccountable Tailor made protocols? Lambda switching Internet weather Hard to quantify Hard to pinpoint bottlenecks

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE February 2005: network transfer test (BWCTL) employing various network monitoring tools involving Jb, Cm, On, Tr, Bologna and JIVE e-VLBI transfer tests

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE e-VLBI transfer tests

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE

Memory-memoryDisk2net-net2diskIn2net- net2disk In2net- net2out UDPTCPUDPTCP Bench via patch Idem, jumbo frames Bench via Amsterdam Idem, jumbo frames Westerbork-JIVE Idem, jumbo frames Bologna-JIVE Jodrell-JIVE Arecibo-JIVE8864 Torun-JIVE Onsala-JIVE JIVE-Haystack61271 Test results

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE POC results Demonstration of feasibility Identification of problems Has led to closer ties with networking community and generated political interest Has laid the foundation for the next step forward (EXPReS)

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE The Future… e-VLBI & EXPReS! I3 proposal to the EC (Communication & Network Development Call) Ranked first out of 43 proposals; nearly fully funded to an amount of 3.9 MEuro. EXPReS = EXpress Production Real-time e-VLBI Service

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE EXPReS aims Upgrade EVN to e-EVN Last mile problem at many telescopes Including connections on a global scale 16 Gb/s lines into Dwingeloo to JIVE correlator Software in field and correlator to become real real-time And look beyond 1 Gb/s More capacity on digital sampling, more bandwidth As being implemented for e-MERLIN in UK Hardware (PC-based) and protocols for transport Correlator with more capacity: distributed correlation

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE Expanding the e-VLBI Network

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE New Antennas 40m antenna at Yebes, Spain Radio and mm frequencies Sardinia Radio Telescope 64m; radio to millimeter

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE Current “e-VLBI” Telescope Status TelescopeStatusCommentsCost (Meuro) JIVE6 x 1 Gbps16 x 1 Gbps upgrade…… WSRT (NL)1 Gbps-- Onsala (SE)1 Gbps10 Gbps upgrade0.15 Torun (PL)1 Gbps- Jodrell Bank & Cambridge (UK) 2.5 Gbps10 Gbps upgrade (7 x 30 Gbps e-MERLIN**) 0.12 Arecibo (USA)155 Mbps1 Gbps upgrade0.4 Medicina (IT)1 GbpsSummer Metsahovi (FI)1 Gbps ** Crucial part of the plan is to combine e-MERLIN and e-VLBI Telescopes transparently.

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE TelescopeStatusCommentsCost (Meuro) Yebes (ES)NYO1 Gbps planned 2006/70.53 VIRAC (LV)NYO1 Gbps planned 2006/7 VLBI Capable 2006 ? 0.53 Shanghai (CN)NYO Gbps planned Last mile: 25 km Poor connectivity between Europe & China (ORIENT) 0.35 Miyun (CN) & Kunming (CN) NYO0.1-1 Gbps planned Last mile: few km 0.3 Urumqi (CN)NYOFibre installed – 2 Mbps Gbps planned 0.1 Hartebeesthoek (SA)NYO1 Gbps planned as part of SKA ambitions Poor connectivity between South Africa & Europe - TIGO (Chile)NYOFibre installed- Noto (IT)NYOLast mile problem3.1 Sardinia (IT)NYOLast mile1.0 Effelsberg (DE)NYOLast mile 5 km but…2.0

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE Towards “real”real-time Reliability of operation Robustness Feedback to stations Status display at stations and Jive Ease of operation Continuous monitoring Better protocols? Software consistency

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE Protocol, buffer selection

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE FABRIC Future Array of Broadband Radio-telescopes on Internet Computing Will need a new correlator for 4 Gb/s Current EVN Mk4 based on 16x16x1Gb/s Implemented on 1024 special chips Next generation will most likely use standard CPUs Current EVN Mk4 processor equivalent 40 T-ops (2bit) LOFAR BlueGene ≈ 27 Tflops 32 station x 4 Gb/s ⇒ 640 T-ops And requirement to route 32x32 4 Gb/s input stream Possible solution: distribute the computing Use the Internet as the cross-switch gets 59 Tflops Proposed to do pilot on the Grid

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE Telescopes will do multicast transmission Compute nodes will be cluster size Transported data may be time/frequency slices Resource allocation from central control Output data stored in central archive, proprietary rights for users Central astronomical scheduling

4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE Conclusions, future POC has been a success, but, progress has been both fast and slow Bottlenecks hard to identify, packetloss does not equate congestion, fairness is a problem, relation packetloss— correlation poorly understood Geant2, Surfnet6 will be hybrid networks lambda switching Bandwidth on demand New protocols, software/hardware interfaces 4—10Gbps transfer—new hardware? e-VLBI is moving ahead; e-EVN will soon be a reality