(bandwidth control) Jeff Boote Internet2 BWCTL (bandwidth control) Jeff Boote Internet2 9 November 2018

What is it? A resource allocation and scheduling daemon for arbitration of iperf tests 11/9/2018

Problem Statement Users want to verify available bandwidth from their site to another. Methodology Verify available bandwidth from each endpoint to points in the middle to determine problem area. 11/9/2018

Typical Solution Run “iperf” or similar tool on two endpoints and hosts on intermediate paths 11/9/2018

Typical road blocks Need software on all test systems Need permissions on all systems involved (usually full accounts*) Need to coordinate testing with others * Need to run software on both sides with specified test parameters * (* bwctl was designed to help with these) 11/9/2018

Functionality (bwctl) Bwctl client application makes requests to both endpoints of a test Communication can be “open”, “authenticated”, or “encrypted” (encrypted is reserved for future) Requests include a request for a time slot as well as a full parameterization of the test Current client is limited in that one of the endpoints must be the localhost, but the protocol is designed to support 3 parties Same “basic” command line options as iperf (some options limited or not implemented.) 11/9/2018

Functionality (bwctld) bwctld on each test host Accepts requests for “iperf” tests including time slot and parameters for test Responds with a tentative reservation or a denied message Reservations by a client must be confirmed with a “start session” message Resource “Broker” Runs tests Both “sides” of test get results Can add a timeout so that “start session” message must be received within a given timeout to thwart DOS 11/9/2018

Scheduling A time slot is simply a time-dependent resource that needs to be allocated just like any other resource. Scheduling is therefore just an extension of the resource allocation model. 11/9/2018

Resource Allocation Model Growing Spheres of control Is the basic parameterization of the requested test allowed? Does the bwctld have enough resources to allow test? Does this host (possibly running other tools) have enough resources? Does this <higher level…> have enough resources? 11/9/2018

Resource Allocation (bwctld) At the bwctld level: Each connection is “classified” (authentication) Each classification is associated with a set of hierarchical limits bwctld.limits 11/9/2018

Architecture 11/9/2018

Demo “mostly” same command-line as Iperf -s and –c options take the remote host as an argument 11/9/2018

Abilene test points available Policy is “in flux” Unclear what resources to allocate to each group Unclear how “limited” resources will be Basic Steps: Locally tested bwctl (key based authentication) Registration of key with Abilene “For now” email ami-key@internet2.edu Run tests and collect data! 11/9/2018

Specific difficulties UDP Iperf doesn’t always send at requested rate Iperf sender hangs (likely Linux/iperf interaction – could be due to signal handling of the bwctl level) End of session is difficult to detect, which is problematic for a “scheduled” timeslot Iperf sometimes takes large amounts of time to finish 11/9/2018

Specific difficulties TCP Large pipe to small pipe Launch a large window Test waits until completion Terminate test to remain within schedule Þ Sets of incomplete tests to interpret Multiple test points with very different delays presents difficulties for window size selection (and other path specific characteristics) bwctl uses the peer to peer server connection to deduce a “reasonable” window If at all possible path specific parameters need to be dynamically configured 11/9/2018

Future Steps Server-less client side for end hosts Integrated tester (based upon Thrulay) 3-party tests (client not on one of the endpoints) Integration with “standard” authentication services Open source development Thrulay is a tool being developed by Stanislav Shalunov that combines achievable bandwidth and rtt delay. 11/9/2018

Availability http://e2epi.internet2.edu/bwctl/ Beta version currently available http://e2epi.internet2.edu/bwctl/ Mail lists: bwctl-users@internet2.edu bwctl-announce@internet2.edu https://mail.internet2.edu/wws/lists/engineering 11/9/2018

11/9/2018