1. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 Networking for the Particle Physics Community - a UK Perspective uIntroduction to the PPNCG uNetwork Topologies uMonitoring Activities uPeering to the US and Canada uGrid Work uSuperJANET4

2. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 Introduction to the PPNCG uMembership includes HEP and Astronomy users –Ensure the community has the required networking facilities –Monitor end-to-end performance –Investigate new network applications / technologies –Provide advice on kit / facilities – URL icfamon.rl.ac.uk/ppncg/title.html uActs as an Interface between HEP and UKERNA –Collaborations: Multicast, H.323, QoS tests, SuperJANET4 uActive Network Monitoring –Uses the ICFA monitoring tools and the PPNCG traceping and ftp –Monitor MANs / SuperJANET & the networks to CERN, DESY, US –Report problems to UKERNA

3. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 SuperJANET Connections in the UK Bishop Grosseteste

4. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 SuperJANET III Backbone

5. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 uSuperJANET – ULCC 3 * 155 Mbit links + 2 * 34 Mbit uPlus 3 * 155 Mbit links u4 * 155 Mbit transatlantic links Oct 00 SuperJANET Connections in London August 2000

6. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 UK to Europe TEN-155 October 2000 uPeering with CANARIE made in London uUses 10 (15) Mbit PVC on the Dante-US line for all Europe u4 * 155 Mbit European links operational

7. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 Monitoring: ping plots UK - CANARIE Ping average round trip time & Lost packet % for icfamon.dl.ac.uk – cepheid.physics.utoronto.ca 15 Oct 00 100 byte packets 15 Oct 00 1000 byte packets

8. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 Monitoring: US Traffic - Trends UKERNA Traffic Kbit/s. Blue Traffic from US Maroon Traffic to US 10 Nov 19991 Nov 2000 UKERNA – US Traffic Daily totals Mbytes Weekend Summer vacation Weekday John Macallister PPNCG Oxford Xmas 99

9. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 Monitoring: ping plots UK - US Ping average round trip time & Lost packet % for 100 byte packets icfamon.dl.ac.uk – ns2.slac.stanford.edu 13 Oct 99 15 Oct 00

10. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 Peering with ESnet uUsed CAR and WRED to give priority to Esnet traffic into JANET. uRtt and packet loss for 100 byte packets. uTests DL to SLAC Start 21 Feb 00 CAR+WRED enabled Config changes to CAR and WRED Further config changes High Priority Normal Priority Robin Tasker PPNCG DL

11. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 Peering with ESnet and Abilene urtt and packet loss for 100 byte packets uPlot start 16 May 00 ESnet High Priority DL - SLAC Abilene High Priority DL - Stanford Further config changes Change in bandwidth restrictions from Abilene into JANET

12. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 The Grid uGovernment has discussed 165M pounds to develop Computing Grid Infrastructure. uAcross Research Councils Initiative – not just Particle Physics u Setting up management structure to co-ordinate PPARC activities. uHave run Globus workshop so US developers could inform the community. uSeveral sites now connected using Globus toolkit. uWeb site www.datagrid.org.uk

13. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 The Grid uStrong involvement in the EU DataGrid middleware proposal. –Proposal accepted – Budget of 9.8M Euros uHEP experiments will need world Grid access. uGrid Networking interests include: –High throughput, reliable, very large data transfers –QoS issues – diffserv, RSVP and packet marking –Monitoring, Metrics, and prediction accuracy –Modelling uPhysics Applications

14. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 The Grid: PAW with Globus Toolkit Linux GASS Server Sun GASS Server SGI GASS Server Campus LAN Zebra HBOOK Toolkit PAW Next Steps: Working on “root” Brokers Data Location Security + Global farms BaBar Tau Analysis Andrew McNab Manchester Demonstrated at the EU IST 2000 Nice Data from Italy & UK

15. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 uATM MAN u7 routers ~0.8ms / router  Router+link 0.4  s/byte uDL-Liverpool ~10 ms rtt - agreement The Grid:Measurements on SuperJANET uSigma ~1.2 ms Latency Mancester - DL Richard Hughes-Jones PPNCG Manchester

16. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 The Grid: Latency on SuperJANET Pink: LAN Blue: MAN-SJ CORE Pink: Man-RAL MB Blue: Man-RAL  ~5  s nice exponential  ~ 2 ms  4.5 -> 6.7 ms

17. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 The Grid: Throughput on SuperJANET Manc – UCL 10 Mbit Manc – Glasgow Manc – RAL 24 Mbit link Manc – RAL 2 Mbit Managed BW

18. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 The Grid: Multi-stream ftp Oxford to SLAC Single stream 8 ftp streams ~ 0.8 single stream John Macallister PPNCG Oxford

19. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 The Grid: Multi-stream ftp SLAC to Oxford

20. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 SuperJANET4 uSuperJANET4 Topology based on Worldcom Trident Network –Tariff: distance independent –SLA: Fix problems not refunds uBackbone initially SDH based 2.5 Gbit (STM-16) uGigabit capacity with growth built in 10 Gbit in 2001, 20 Gbit in 2002 uLink existing Regional Networks uMigration to DWDM during 2001 uTechnology review in 2003

21. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 SuperJANET4: Backbone and Access links uWorldcom supply the transmission: –SDH –Circuit protection – fallback in 50 ms uUKERNA overlay the IP: –Packet over SDH –Core PoP IP routers at Worldcom –Backbone Access Router at MANs –Routers managed by UKERNA NOSC uAccess Link Fibers: –Installed by Worldcom –Large MAN 2.5 Gbit -> 10-20 Gbit –Medium MAN 622 Mbit -> 2.5 Gbit u4 node DWDM development net

22. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 SuperJANET4: Worldcom High Level Design STM16 STM 4 STM leased Dark Fibre STM 64 WC Metro DWDM

23. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 SuperJANET4: Routing uCISCO 12016 routers from Logical uCore PoP –Responsible for switching packets –No “edge” functions – congestion avoidance –Symmetric L3 routing – eBGP peering to BAR –Packet over SDH Backbone C-PoP SDH ADM Regional MAN BAR Access Links eBGP Peering uBackbone Access Router –Provide filtering – drop if src. not from MAN –Traffic management – classify packets –Interface to MANs with Gigabit Ethernet uMulticast –Integral part of SuperJANET4 – not tunneled –Run sparse-mode PIM

24. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 SuperJANET4: MAN Development uSupport for Router and Link upgrades uIncrease: –Capacity 1 Gbit now 10-20 Gbit in 2 years –Resilience – 99.95 % uptime –Reach – additional sites & schools uNetNorthWest as an example NNW SuperJANET4 BAR Manchester Liverpool Preston SDH Gigabit Ethernet SDH Trunk uSDH trunks 622 Mbit, will be increased with traffic load

25. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 SuperJANET4: Network Services uServices co-located at CPoPs or BARs –Provides High BW access uCentrally managed by UKERNA –Remote console access –Remote power management –Network and access control provided uProvider responsible for service not network uExamples: –DNS servers –www caches –usnet servers –Video servers - content from academic / external –Videoconference H.323 MCUs Backbone Regional MAN BAR C-PoP Access Links Server SDH ADM Server

26. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 SuperJANET4: Project Update uRollout Status: –Backbone in place at Worldcom C-PoPs – SDH resilience shown. –Routers at Logical UK, acceptance in progress, delivery 15 Nov 00 –Manchester-ULCC traffic live on 5 Dec 00 –All sites Using SuperJANET4 by 31 Mar 01 uNetwork Users –HE sites – e.g. Universities already on SuperJANET III –FE sites – e.g. Colleges almost all connected at 2 Mbit –Schools – Further study in progress uChallenges –Quality of Service Priority traffic – packet marking & WRED Jitter sensitive traffic H.323 – Low latency queuing MDRR –MPLS VPNs

27. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 Cisco Router Acceptance Testing

28. R. Hughes-Jones Advanced Networks Workshop Montreal November 2000 Conclusion & Status uNetwork Links to US and Europe are OK (at the moment !) uPeering with Abilene, ESnet and CANARIE OK, interesting effects with CAR – WRED ! uConsiderable UK interest in the Internet2 tests uGrid: active in several areas including networks uSuperJANET4 rollout is on target