WLCG: Are we ready for IPv6? David Kelsey (STFC-RAL) ISGC 2014, Taipei 26 Mar 2014

Co-authors Thanks to all my colleagues S. Campana (CERN), K. Chadwick (FNAL), G. Chen (IHEP), J. Chudoba (FZU), A. Dewhurst (STFC), M. Elias (FZU), S. Fayer (Imperial), T. Finnern (DESY), C. Grigoras (CERN), B. Hoeft (KIT), T. Kouba (FZU), F. Lopez Munoz (PIC), E. Martelli (CERN), M. Mitchell (Glasgow), K. Ohrenberg (DESY), A. Pfeiffer (CERN), F. Prelz (INFN), F. Qi (IHEP), D. Rand (Imperial), S. Rozsa (Caltech), A. Sciaba (CERN), R. Voicu (Caltech), C. J. Walker (QMUL), T. Wildish (Princeton ) 26/03/2014WLCG IPv62

Outline Update since ISGC 2013 HEPiX IPv6 working group IPv4 address exhaustion – global and at CERN File transfer tests Other news Software and Tools survey Plans for /03/2014WLCG IPv63

HEPiX IPv6 Working Group Consider how IPv6 should be deployed in HEP – Worldwide Large Hadron Collider Grid (WLCG) Readiness and Gap analysis HEP applications, middleware, security issues, system management and monitoring tools, end to end network monitoring tools, operations Run a distributed HEP testbed – to help explore all the above issues 26/03/20144WLCG IPv6

26/03/2014WLCG IPv65 Global IPv4 free addresses …

IPv4 free addresses - /8 26/03/2014WLCG IPv66

IPv4 Addresses From Geoff Huston ( ) IANA Unallocated Address Pool (Global) Exhaustion happened:03-Feb-2011 Projected Regional (RIR) Address Pool Exhaustion Dates: – APNIC: 19-Apr-2011 (Asia Pacific - happened) – RIPENCC: 14-Sep-2012 (Europe - happened) – LACNIC: 06-Oct-2014 (South America) – ARIN: 31-Mar-2015 (North America) – AFRINIC: 11-Jun-2020 (Africa) 26/03/2014WLCG IPv67

IPv4 RIR /8 blocks left 26/03/2014WLCG IPv68

Google accessed by IPv6 clients 26/03/2014WLCG IPv69 Globally 3.4 % Belgium15.6% USA 6.5% Taiwan 0.6%

26/03/2014WLCG IPv610 IPv4/IPv6 at CERN … IPv4 addresses predicted to run out at CERN during 2015 And Run 2 starts “soon”

Network Database: Schema and Data IPv6 Ready Configuration Manager supports IPv6 routing Admin Web: IPv6 integrated 2013 The Data Center is Dual-Stack Gradual deployment on the routing infrastructure starts NTPv6 and DNSv6 Today DHCPv6 Infrastructure is Dual-Stack Firewallv6 automated configuration User Web and SOAP integrate IPv6 Automatic DNS AAAA configuration CHEP2013 talk by David GUTIERREZ RUEDA – “Network architecture and IPv6 deployment at CERN” IPv6 deployment at CERN 26/03/2014 WLCG IPv6

26/03/2014WLCG IPv612 File transfer tests over IPv6 …

The HEPiX IPv6 Testbed A dedicated distributed testbed Connected to IPv6 and IPv4 networks – IPv6-only/IPv4-only names also registered in DNS – e.g. hepix-v6.desy.de & hepix-v4.desy.de Caltech, CERN, Chicago, DESY, FNAL, FZU, Glasgow, Imperial, IHEP, INFN, KIT, PIC, QMUL 26/03/201413WLCG IPv6

IPv6 file transfer tests Tony Wildish (CMS) Simple mesh of simultaneous data transfers (IPv6) Transfers a 1 GB file using GridFTP (globus-url-copy) – Measures time to transfer – Records any errors Uses UberFTP to confirm arrival and then delete Have been running for last 8 months 26/03/2014WLCG IPv614

GridFTP IPv6 data transfer mesh 26/03/2014WLCG IPv615 > 2 PB data transferred in 6 months Success rate > 87% Very High! Operating at risk Time to transfer max 500 s

Data transfer lessons learned CMS PhEDEx is production ready on IPv6 The testing is very useful for finding site IPv6 problems – E.g. allows some level of stress testing Over the months we have experienced some breaks, performance problems, asymmetric routing, … – E.g. cpu utilisation in FNAL border router reaches 60% with IPv6 traffic flows of Mbps The reason for ongoing data transfer tests It is useful for a site to be in the IPv6 testbed 26/03/2014WLCG IPv616

26/03/2014WLCG IPv617 Other news …

Dual-Stack services at Imperial Tier 2 Imperial College, London Configured a subset of their service to be IPv4/IPv6 dual stack – DNS, SSH, NFS, EMI 2 and EMI 3 CREAM CEs, EMI 2 Worker Nodes, ARC CE and dCache (headnode, SRM component only), all BDII services – Stateless address autoconfiguration (SLAAC) No problems observed – Did not need to turn IPv6 off 26/03/2014WLCG IPv618

dCache & IPv6 Pre-release of dCache tested at NDGF – IPv6 fixes will not be backported to V2.6 Seems to work – BUT all Gridftp-1, thus proxied via the FTP door instead of the data being sent from client directly to the pool dCache uses the response to the PASV FTP command to steer data connections to a pool of servers for load balancing – And obeying RFC2428 which says protocol and host fields of the EPSV response "MUST be blank” (for NAT) But Globus, jGlobus, Uberftp clients *do* parse these fields Discussions ongoing with dCache team 26/03/2014WLCG IPv619

26/03/2014WLCG IPv620 Software and Tools survey …

26/03/2014WLCG IPv621

Software survey (2) 26/03/2014WLCG IPv622

Software survey (3) 26/03/2014WLCG IPv623

Software survey (4) 26/03/2014WLCG IPv624

26/03/2014WLCG IPv625 Future plans and next steps …

Next testing KIT, PIC, FZU, NDGF – PhEDEx/SRM/dCache tests (on testbed) Then move to use of SRM/FTS/PhEDEx – With various production endpoints – Volunteer sites Working on clearly defined use case (from expts) – CERN Worker Node/Virtual Machine IPv6-only – What WLCG services outside CERN need to be dual-stack? 26/03/2014WLCG IPv626

ATLAS testing Alastair Dewhurst (ATLAS, RAL) AGIS – Flags added for IPv6 support on storage endpoints Start with simple data movement tests – globus_url_copy, lcg commands Then data tests using HammerCloud Frontier service – Squid 2.8 is non IPv6 compatible Squid 3.1 at Tier 2s should be usable 26/03/2014WLCG IPv627

Next steps Perform survey of all WLCG sites – When will they run out of IPv4 addresses – When will they be capable of IPv6 support One-day pre-GDB meeting on IPv6 – 10 June 2014 at CERN – Aimed primarily at Tier 1 sites (others welcome) – Encourage IPv6 support – Move to dual-stack WLCG services – Testing needs to move to production services 26/03/2014WLCG IPv628

Further info HEPiX IPv6 web HEPiX IPv6 wiki Working group meetings WLCG Operations IPv6 Task Force Paper accepted for publication in proceedings of CHEP /03/201429WLCG IPv6

Summary Good progress is being made – proving that dual stack IPv4/IPv6 services work in WLCG – But work is slow and there is lots more to do! We are very much effort-limited – Tier 1 and Tier 2 sites – Experiments We do need to encourage sites to deploy IPv6 – The June 2014 workshop should help here VOLUNTEERS to join the group are always welcome – please contact me! 26/03/201430WLCG IPv6

26/03/2014WLCG IPv631 Questions ?