1. ESLEA and HEPs Work on UKLight Network

2. ESLEA Exploitation of Switched Lightpaths in E- sciences Applications Exploitation of Switched Lightpaths in E- sciences Applications Multi-disciplined Multi-disciplined Protocol Development Protocol Development Exploitation by HEP ( ATLAS and CDF), Radio- astronomers, E-Health, HPC Exploitation by HEP ( ATLAS and CDF), Radio- astronomers, E-Health, HPC Using dedicated point-to-point light path channels on research UKLight Network for R&D purposes Using dedicated point-to-point light path channels on research UKLight Network for R&D purposes Bulk Data Transfers / Circuit Reservation and deployment /Transport Protocols / Real Time Visualization Bulk Data Transfers / Circuit Reservation and deployment /Transport Protocols / Real Time Visualization

3. HEP Connections RAL-CERN RAL-CERN UCL-Fermilab UCL-Fermilab Lancaster-Edinburgh Lancaster-Edinburgh RAL-Lancaster RAL-Lancaster SARA-Lancaster SARA-Lancaster Lancaster-Manchester Lancaster-Manchester

4. Lancaster Edinburgh Objectives Investigate the use of an alternate (in this case UDT) protocol to maximise the potential of an optical circuit Investigate the use of an alternate (in this case UDT) protocol to maximise the potential of an optical circuit Utilise this protocol in such a was as to be of practicable use to users of the grid. Utilise this protocol in such a was as to be of practicable use to users of the grid.

5. What is UDT ? UDT: UDP-based Data Transfer Protocol UDT: UDP-based Data Transfer Protocol Application level, end-to-end, unicast, reliable, connection-oriented, data transport protocol. Application level, end-to-end, unicast, reliable, connection-oriented, data transport protocol. Approximately 90% utilisation Approximately 90% utilisation of available bandwidth of available bandwidth

6. Servers Hardware : Hardware : Dual Xeon 3.2GHz dual core Dual Xeon 3.2GHz dual core 2 GB RAM 2 GB RAM Dual PCI-X bus Dual PCI-X bus 2 x Gigabit Ethernet 2 x Gigabit Ethernet SATA Raid controller SATA Raid controller 6 x SATA disks 6 x SATA disks 1 x SATA system disk 1 x SATA system disk OS OS Scientific Linux 3.0.5 with 2.4.21 Scientific Linux 3.0.5 with 2.4.21

7. Disk Testing Results Ext3 Write Speed / CPU Load Ext3 Read Speed / CPU Load XFS Write Speed / CPU Load XFS Read Speed / CPU Load System Disk 320 Mb/s (20%) 336 Mb/s (10%)-- Raid 0 (write caching disabled) 1544 Mb/s (96%) 1304 Mb/s (33%) 2160 Mb/s (63%) 1304 Mb/s (34%) Raid 0 (write caching enabled) 1143 Mb/s (70%) 1221 Mb/s (30%) 1184 Mb/s (33%) 1124 Mb/s (28%) Raid 5 (write caching disabled) 90 Mb/s (6%) 363 Mb/s (10%) 97 Mb/s (3%) 348 Mb/s (10%) Raid 5 (write caching enabled) 905 Mb/s (60%) 1222 Mb/s (30%) 1001 Mb/s (30%) 1216 Mb/s (30%)

8. Network Testing Tests were performed with default kernel and application settings and then again after applying changes to maximise network speeds Tests were performed with default kernel and application settings and then again after applying changes to maximise network speeds BDP for this link should be : BDP for this link should be : BDP = Bandwidth (MB/s) * RTT (seconds) BDP = Bandwidth (MB/s) * RTT (seconds) BDP = (1 * 1024 / 8) * (0.3 / 1000) BDP = (1 * 1024 / 8) * (0.3 / 1000) BDP = 0.0384 MB (39.32KB) BDP = 0.0384 MB (39.32KB)

9. Basic Network Test results Protocol Out of the box throughput Maximum Throughput TCP 650 Mb/s 940 Mb/s UDP 957 Mb/s UDT 950 Mb/s

10. File Transfer Test Results Transfer Program Out of the box tuning Maximal Throughput ftp 565 Mb/s 820 Mb/s GridFTP 570 Mb/s 850 Mb/s UDT test application 460 Mb/s 935 Mb/s

11. What next ? The Basic Network tests and the File transfer tests need to be re-performed once the UKLight link between Lancaster and Edinburgh is fully functional The Basic Network tests and the File transfer tests need to be re-performed once the UKLight link between Lancaster and Edinburgh is fully functional Integration of UDT into a functional GridFTP server and client Integration of UDT into a functional GridFTP server and client Deployment of modified software into test LCG sites. Deployment of modified software into test LCG sites.

12. Lancaster RAL Link T1-T2 transfer testing T1-T2 transfer testing Avoid production network induced bottlenecks Avoid production network induced bottlenecks Firewall @ RAL Firewall @ RAL Internal LAN traffic Internal LAN traffic Tested using : Tested using : Command line srmcp in shell script Command line srmcp in shell script FTS controlled transfers FTS controlled transfers

13. Achieved Achieved Peak of 948Mbps Peak of 948Mbps Transferred: Transferred: 8TB in 24 hours - 800+ Mbps aggregate rate 8TB in 24 hours - 800+ Mbps aggregate rate 36TB in 1 week - 500+ Mbps aggregate rate 36TB in 1 week - 500+ Mbps aggregate rate Over 800Mbps when running, but 0Mbps in downtimes a problem Over 800Mbps when running, but 0Mbps in downtimes a problem Parallel file transfers increase rate Parallel file transfers increase rate Better utilisation of bandwidth Better utilisation of bandwidth Staggered initialisation of transfers reduces overhead from initialisation/cessation of individual transfers. Rate increase from 150Mbps to 900Mbps Staggered initialisation of transfers reduces overhead from initialisation/cessation of individual transfers. Rate increase from 150Mbps to 900Mbps 2% (18Mbps) reverse traffic flow for 900Mbps transfer 2% (18Mbps) reverse traffic flow for 900Mbps transfer

14. FTS transfers not yet as successful as srmcp only transfers FTS transfers not yet as successful as srmcp only transfers Greater overheads? Greater overheads? More optimisation needed More optimisation needed Single FTS file transfer gives 150Mbps Single FTS file transfer gives 150Mbps Same as srmcp Same as srmcp Concurrent FTS file transfers scales at lower rate than srmcp Concurrent FTS file transfers scales at lower rate than srmcp All single stream transfers All single stream transfers FTS tests currently used single source file FTS tests currently used single source file Srmcp used with multiple source files Srmcp used with multiple source files Rate varies dependent on direction Rate varies dependent on direction Possibly explained by difference in dCache setup Possibly explained by difference in dCache setup V0 Dependency V0 Dependency kernel settings kernel settings Disk I/O limitations Disk I/O limitations SRM pool load balancing SRM pool load balancing To be investigated To be investigated File size affects rate of transfer File size affects rate of transfer Single stream rate varies 150 to 180 Mbps with increase from 1 to 10 GB file size Single stream rate varies 150 to 180 Mbps with increase from 1 to 10 GB file size

15. Lancaster SARA Link Link not yet active Link not yet active Tests similar to Lancaster-RAL and Lancaster-Edinburgh Tests Tests similar to Lancaster-RAL and Lancaster-Edinburgh Tests Bulk File Transfers Bulk File Transfers UDT Protocol Testing UDT Protocol Testing Study of effect of International/Extended link length Study of effect of International/Extended link length SARA storage capacity underused, RAL capacity currently too small for UK simulation storage SARA storage capacity underused, RAL capacity currently too small for UK simulation storage Also, SARA to test ATLAS Tier1 fallback scenario (FTS catalogues etc.) Also, SARA to test ATLAS Tier1 fallback scenario (FTS catalogues etc.) Are we capable of connecting to an alternate Tier1? Are we capable of connecting to an alternate Tier1?

16. Lancaster Manchester Link Intra-Tier2 site Testing Intra-Tier2 site Testing Homogeneous Distributed Tier2 Homogeneous Distributed Tier2 dCache Head node at Lancaster, pool nodes at both Lancaster and Manchester dCache Head node at Lancaster, pool nodes at both Lancaster and Manchester Test Transfers to/from RAL Test Transfers to/from RAL Test of Job submission to close CE/WNs Test of Job submission to close CE/WNs Possible testing of xrootd within dCache Possible testing of xrootd within dCache

17. www.eslea.uklight.ac.uk www.eslea.uklight.ac.uk Connecting to UKLight Connecting to UKLight Documents Documents