Optimisation across networks Dyalog User Conference 18 th October 2012 Paul Grosvenor Managing Director.

Slides:



Advertisements
Similar presentations
Switching Techniques In large networks there might be multiple paths linking sender and receiver. Information may be switched as it travels through various.
Advertisements

Internet Protocol How does information get sent from one device to another across a WAN?
CCNA3: Switching Basics and Intermediate Routing v3.0 CISCO NETWORKING ACADEMY PROGRAM Switching Concepts Introduction to Ethernet/802.3 LANs Introduction.
CE363 Data Communications & Networking Chapter 7 Network Layer: Internet Protocol.
Umut Girit  One of the core members of the Internet Protocol Suite, the set of network protocols used for the Internet. With UDP, computer.
XCP: Congestion Control for High Bandwidth-Delay Product Network Dina Katabi, Mark Handley and Charlie Rohrs Presented by Ao-Jan Su.
Receiver-driven Layered Multicast S. McCanne, V. Jacobsen and M. Vetterli SIGCOMM 1996.
By Aaron Thomas. Quick Network Protocol Intro. Layers 1- 3 of the 7 layer OSI Open System Interconnection Reference Model  Layer 1 Physical Transmission.
1 K. Salah Module 2.1: QA – Putting it all together What is the max number of users/connections/sessions a particular network can support for handling.
Multiple constraints QoS Routing Given: - a (real time) connection request with specified QoS requirements (e.g., Bdw, Delay, Jitter, packet loss, path.
Data Communications Architecture Models. What is a Protocol? For two entities to communicate successfully, they must “speak the same language”. What is.
Computer Networks Transport Layer. Topics F Introduction  F Connection Issues F TCP.
1 K. Salah Module 6.1: TCP Flow and Congestion Control Connection establishment & Termination Flow Control Congestion Control QoS.
TCP. Learning objectives Reliable Transport in TCP TCP flow and Congestion Control.
Problems.
Chapter 23: ARP, ICMP, DHCP IS333 Spring 2015.
Networking. Protocol Stack Generally speaking, sending an message is equivalent to copying a file from sender to receiver.
Network Measurement Bandwidth Analysis. Why measure bandwidth? Network congestion has increased tremendously. Network congestion has increased tremendously.
Ch. 28 Q and A IS 333 Spring Q1 Q: What is network latency? 1.Changes in delay and duration of the changes 2.time required to transfer data across.
Top-Down Network Design Chapter Two Analyzing Technical Goals and Tradeoffs Copyright 2010 Cisco Press & Priscilla Oppenheimer.
TCP Behavior across Multihop Wireless Networks and the Wired Internet Kaixin Xu, Sang Bae, Mario Gerla, Sungwook Lee Computer Science Department University.
CS332 Ch. 28 Spring 2014 Victor Norman. Access delay vs. Queuing Delay Q: What is the difference between access delay and queuing delay? A: I think the.
TRANSPORT LAYER T.Najah Al-Subaie Kingdom of Saudi Arabia Prince Norah bint Abdul Rahman University College of Computer Since and Information System NET331.
CS448 Computer Networking Chapter 1 Introduction to Computer Networks Instructor: Li Ma Office: NBC 126 Phone: (713)
ISO Layer Model Lecture 9 October 16, The Need for Protocols Multiple hardware platforms need to have the ability to communicate. Writing communications.
These materials are licensed under the Creative Commons Attribution-Noncommercial 3.0 Unported license (
Chapter 2 – X.25, Frame Relay & ATM. Switched Network Stations are not connected together necessarily by a single link Stations are typically far apart.
S3C2 – LAN Switching Addressing LAN Problems. Congestion is Caused By Multitasking, Faster operating systems, More Web-based applications Client-Server.

Dr. John P. Abraham Professor UTPA
Chapter 15 – Part 2 Networks The Internal Operating System The Architecture of Computer Hardware and Systems Software: An Information Technology Approach.
DCCP, TFRC & Open Problems in Congestion Control for Media Applications Tom Phelan 13-Feb-2007 ICCRG.
Three Blind Mice Dyalog User Conference 17 th October 2012 Paul Grosvenor Managing Director.
1 Message Delineation Identify the start and end of messages by: Identify the start and end of messages by:
Networking Fundamentals. Basics Network – collection of nodes and links that cooperate for communication Nodes – computer systems –Internal (routers,
Network Performance. Performance (1) What would be the characteristics of the ideal network? –It would be completely transparent in every conceivable.
Forward Error Correction vs. Active Retransmit Requests in Wireless Networks Robbert Haarman.
1 CS 4396 Computer Networks Lab TCP – Part II. 2 Flow Control Congestion Control Retransmission Timeout TCP:
Welcome from Optima Systems COSMOS performance improvements Paul Grosvenor Deerfield Beach 2013 Tuesday October 22nd.
What is the Speed of the Internet? Internet Computing KUT Youn-Hee Han.
Chapter 11.4 END-TO-END ISSUES. Optical Internet Optical technology Protocol translates availability of gigabit bandwidth in user-perceived QoS.
TCP continued. Discussion – TCP Throughput TCP will most likely generate the saw tooth type of traffic. – A rough estimate is that the congestion window.
Optimization Problems in Wireless Coding Networks Alex Sprintson Computer Engineering Group Department of Electrical and Computer Engineering.
Networks, Part 2 March 7, Networks End to End Layer  Build upon unreliable Network Layer  As needed, compensate for latency, ordering, data.
TCP Traffic Characteristics—Deep buffer Switch
Data Link Layer. Data link layer The communication between two machines that can directly communicate with each other. Basic property – If bit A is sent.
TCP/IP1 Address Resolution Protocol Internet uses IP address to recognize a computer. But IP address needs to be translated to physical address (NIC).
Network Layer. application transport network link physical message segment packet frame signal Network Architecture.
Ch. 28 Q and A CS 332 Spring A little quiz Q: What is network latency? 1.Changes in delay and duration of the changes 2.time required to transfer.
Chapter 28 Q and A IS 333 Spring A quiz question Q: What is network latency? 1.Changes in delay and duration of the changes 2.time required to transfer.
UDP: User Datagram Protocol. What Can IP Do? Deliver datagrams to hosts – The IP address in a datagram header identify a host – treats a computer as an.
Chapter 9: Transport Layer
More on WANs Fiber Optic Cables Used in Internet backbone
Neha Jain Shashwat Yadav
Vocabulary Prototype: A preliminary sketch of an idea or model for something new. It’s the original drawing from which something real might be built or.
Speaker : Che-Wei Chang
Process-to-Process Delivery, TCP and UDP protocols
PART 5 Transport Layer Computer Networks.
Vocabulary Prototype: A preliminary sketch of an idea or model for something new. It’s the original drawing from which something real might be built or.
Made by:- Naveen singh & Mazaharul Haque
Transport Layer Unit 5.
Dr. John P. Abraham Professor UTPA
Dr. John P. Abraham Professor UTRGV, EDINBURG, TX
File Transfer Issues with TCP Acceleration with FileCatalyst
Dr. John P. Abraham Professor UTPA
TRANSMISSION CONTROL PROTOCOL
Net 323 D: Networks Protocols
Beyond FTP & hard drives: Accelerating LAN file transfers
Transport Layer 9/22/2019.
Impact of transmission errors on TCP performance
Presentation transcript:

Optimisation across networks Dyalog User Conference 18 th October 2012 Paul Grosvenor Managing Director

Insert image here

In this conference we have talked about parallel processing, code optimisation, interpreter improvements etc. Speed is clearly an important feature of our daily lives We have spent little time looking at the infrastructure that all our codes inhabits

So why am I even talking on this subject today ? Because often we do not give proper consideration of the impact of our application on the network

So let us look at some very basic scenarios

Common misconceptions The Server is always up The server is dedicated to me The server has no space limitations The network bandwidth is infinite The client computers are really, really fast The network has zero latency We have perfect Infrastructure

Infrastructure

January 2011

March 2011

June 2012

fExist 'D:\APL\MyFile.DCF' 1 'D:\APL\MyFile.DCF' ⎕ ftie 0 FILE NAME ERROR: Unable to open file ("The system cannot find the file specified.") 'D:\APL\MyFile.DCF' ⎕ FTIE 0 ∧ fExist 'D:\APL\MyFile.DCF' 0 'D:\APL\MyFile.DCF' ⎕ ftie 0 1 So what ?

Excerpts from MSDN Be aware that in some cases Explorer looks at the metadata of each file in a folder when you open a folder, which requires a read of each and every file within it.

Excerpts from MSDN 2,000-3,000 files [in a directory] is the limit of my patience

Excerpts from MSDN The core Windows API is highly optimized for this basic scenario, and returns these attributes by default along with the rest of the file information. However, the.NET Framework doesn’t take advantage of this data, and instead goes back to the network location to ask for all of the file attributes. This chatty behaviour adds a handful of network round trips for each file or directory, making the directory listing many times slower: hundreds or thousands of times slower in many cases. The Framework team addressed this as part of.NET 4.0.

Problems at the speed of light A satellite in geosynchronous orbit imposes a path length of at least km between transmitter and receiver. This leads to a minimum latency of 473 ms NASA’s deep space network has latencies measured in minutes

Goodput vs Throughput The number of useful bits of information transmitted at the application level over time In June 2001 Zona Research showed that an interaction delay of more than 8 seconds (over the internet) became rapidly intolerable IBM suggested a typical GUI response of under 4 seconds to be acceptable depending upon the application (I think) These days a user would expect sub 1 second response

Transmission overhead Channel speed 100Mb/s Max packet size 1500 bytes + 26 bytes 20 bytes IP header 20 bytes TCP header 12 byte time stamp (Unix, Linux, Mac) Leaving 1460 bytes per packet available Goodput = 1460 / 1526 x 100 = 95.6Mb/s So we lose 5% before we even start

Now what ? Collisions Lost packets Packet re-transmission Interframe gap Packet queuing delay Flow control Processing delay Congestion avoidance

Oh my god – What else ? Security protocols on the server Updates on client machine Updates on server machine Signal / noise ratio The list just keeps going on …

The reality is that unless you have very specialised equipment running in ideal environments you will be lucky to achieve a Goodput better than 70% of expectations

So what can I do Never forget your expected infrastructure limitations Avoid deep directory structures Keep the files in a single directory to a sensible number (experiment) – 1,000 fine – 10,000 probably OK – 100,000 probably not

So what can I do Consider file size; a number of smaller files may perform better than a single large file Reduce junk traffic Remember that other people use your server as well Simplify your security structure if you can – Application level not directory or file level

Your server and network are analogous to a heart and vascular system Look after them, treat them with respect, and hopefully they will look after you

Thank You Contact us: Optima House, Mill Court, Spindle Way, Crawley, West Sussex RH10 1TT Tel: Fax: