Proposed Diffserv Evaluation for Emerge Sites Volker Sander Argonne National Laboratory 1/20/2000

2 Outline l Basic GARA overview l New GARA-features l Basic set of experiments l Installation of GARA l Running the experiments using GARA l Timeline

3 The Basic Diffserv Concept Aggregate based scheduling implements EF PHB Classify and Mark packets Shape Traffic Flow based Classify and Mark packets Shape Traffic

4 Proposed Implementation Client GARA API GARA Diffserv Resource Manager Server

5 The Multidomain Extension Server GARA Diffserv Resource Manager Client GARA API GARA Diffserv Resource Manager

6 GARA Globus Architecture for Reservation and Allocation l Three important contributions: –Support for Advance Reservations (AG) –Uniform treatment of underlying resources >Network, disk, CPU etc… >Currently supported: Differentiated Services, DPSS, and DSRT –Layered architecture enables generic co- reservation agents >User is able to select API best suitable for his need

7 GARA Basics—Reservations l There is a generic “reservation”, which has several properties: –Start Time (“now” or future) and Duration –Resource type/Underlying resource identifier –Resource-specific (bandwidth, % CPU…) l All reservations are treated uniformly: –Create/Modify (Given properties) => Returns Reservation Handle –Destroy –Monitor (Callbacks or Polling)

8 A picture of what happens (creating a reservation) Gatekeeper (Authenticate and Authorize) Create User Program or Agent Reservation Handle Resource Manager Gatekeeper Service Local Resource Manager API (LRAM) Resource (Router...)

9 A picture of what happens (claiming a reservation) User Program bind(handle) LRAM API Resource Manager Setup reservation Resource Enforce Reservation Note: GARA supports third party binding

10

11 Example: Network reservation l In advance, make a reservation:  RSL: & (reservation-type=network) (start-time=now) (duration=3600) (endpoint-a= ) (endpoint-b= ) (bandwidth=1000) (protocol=udp) (reservation-subtype=low-latency)  Handle: Opaque string H 1 l At run-time: –Bind(H 1,RSL)  RSL: & (which-endpoint=a) (endpoint-a-port=5000) (endpoint-b-port=5000)

12 The Resource Manager l To track advanced reservations, we use a simple “slot table” l We assume exclusive access to resource through the resource manager l We aren’t tied to this particular resource manager % of available resource allocated 0% 100% Time

13 End-to-End Network Reservations l Algorithm for a single network reservation –Discover which gatekeepers/resource managers need to be contacted (MDS) –Make reservation at each one. User process Workstation Resource Manager MDSEnd-to-End Co-Reservation Agent

14 The GARA API l globus_gara_reservation_create()  Gatekeeper contact  RSL reservation specification  Reservation Handle or Error l globus_gara_reservation_modify()  Old Reservation Handle  RSL reservation specification  New Reservation Handle or Error

15 The GARA API (continued) l globus_gara_reservation_cancel()  Reservation Handle  Error l globus_gara_reservation_status()  Reservation Handle  Status/Error l globus_gara_reservation_callback_register()  Reservation Handle  Callback Function/User Parameter  Error

16 New GARA features l New Reservation-Subtypes –low-latency >Used to support jitter sensitive applications using the same EF aggregate behavior as high-throughput TCP applications >Based on traffic shaping and Priority-Queuing >Currently being evaluated –background >Used to support bulk transfers, including deadline support. Takes at least the amount of premium traffic required to fulfill the deadline; more if not used actively l Monitoring of the network edges –Provides feedback to applications when they send too fast

17 Network + CPU Reservations

18 Example of bulk transfer (LAN)

19 Example of bulk transfer (WAN)

20 Feedback from Monitoring the Edges

21 Evaluation Tools l UDP Traffic Generator –Modified Version of Andy Adamson’s gen_send and gen_recv >Evaluate admission control >Creating competing traffic –MGEN/Drec >Evaluate delay and Jitter for Premium UDP Flow l Modified Version of ttcp –GARA-enabled (wait for reservation) –Support for a desired application rate >long term control >short term control –Consecutive bandwidth reporting –Bulk transfer ttcp

22 Basic Experiments l Evaluate Admission Control l Evaluate premium TCP Flows under Congestion –Analyze network traces with tcpdump l Evaluate UDP –Delay –Jitter l Evaluate Inter-Domain Issues Fundamental idea: Get the necessary knowledge to be able to experiment with REAL applications

23 Basic Experiment I l Goal: Proof of Admission Control –Gather packet statistics from gen_recv for different configurations and different traffic profiles:

24 Basic Experiment I gen_send GARA Diffserv Resource Manager gen_recv

25 Basic Experiment II l Goal: Proof of Premium Quality for TCP flows –Run modified ttcp with different traffic and reservation profiles; gather receiver bound statistics –Create congestion using gen_send/recv

26 Basic Experiment II l Enhancement –Use tcpdump and tcptrace get a deep overview on the behavior –Use netlogger_enabled ttcp_gara and gen_recv to publish data into MDS. “Real Time Visualization”

27 Basic Experiment II GARA Diffserv Resource Manager ttcp -t ttcp -r gen_sendgen_recv

28 Basic Experiment III l Goal: Demonstrate Premium Quality for UDP flows –Run mgen/drec with different traffic and reservation profiles; gather receiver bound statistics –Create congestion using gen_send/recv

29 Basic Experiment III GARA Diffserv Resource Manager mgen drec gen_sendgen_recv

30 Basic Experiment IV l Goal: Proof Premium Quality for UDP flows sharing one EF aggregate –Run mgen/drec with different traffic and reservation profiles; gather receiver bound statistics –Run high-throughput ttcp using a correct reservation –Create congestion using gen_send/recv

31 Basic Experiment IV GARA Diffserv Resource Manager ttcp -t ttcp -r mgendrecgen_recvgen_send

32 Additional Experiments l Bulk-Transfer l Reevaluate inter-domain behavior –Add Traffic-Shaping between domains l Test Inter-Domain Admission control l Prepare the Instrumentation l Go with real applications

33 Installing GARA l Get GARA –(currently through from l Get SSLeay and install it l Get Globus 1.1 configure and compile it deploy is not necessary for doing tests but recommended Run grid-cert-request to apply for a Grid X509 Certificate add “gara-service stderr_log,local_cred - $GARA_DIR/gara/programs/globus_gatekeeper_gara_service -d” to $GLOBUS_DEPLOY_DIR/etc/globus-services l Read gara/docs/* l Edit gara/build l Run gara/build l Customize GARA –gara/resource_manager/programs/diffserv_manager.conf –gara/resource_manager/programs/setup_flow.cfg –(Samples are in the documentation directory)

34 Building GARA l $GARA_DIR/build l #!/bin/ksh l CC=cc l CXX=CC l CFLAGS="-g -v -mt" l GLOBUS_DIR=/soft/pub/packages/globus-1.1.0/development/sparc-sun-solaris2.7_pt l hreads_standard_debug l SSL_LIBRARY_DIR=/soft/pub/packages/SSLeay-0.9.0/lib l export CC CXX CFLAGS GLOBUS_DIR SSL_LIBRARY_DIR l configure l make clean l make depend l make all l # l # For portability reasons you might not want to do a make withdsrt.. l # in case: just comment out this line. l #make withdsrt l Run build

35 Configuring GARA l resource_manager/programs/diffserv_manager.conf PublicationMethod web mds LDAPDistinguishedName "dslnet1.mcs.anl.gov:34880:/C=US/O=Globus/O=Argonne Nati onal Laboratory/OU=MCS/CN=roy” MDSPublishPassword “password” WEBPublishFile /home/sander/public_html/diffserv_manager_slots.html NoOfRouters 2 IPAddressesServed[1] , , , , , , IPAddressesServed[2] , , , , , l Parameters: "LDAPDistinguishedName", "MDSPublishPassword", "WEBPublishFile", "FILEPublishFile", "Quantity", "ClearSlotTable", "SlotTableFilename", "Verbose", "PublicationMethod", "DontReallyBindReservations"

36 Configuring GARA l resource_manager/programs/setup_flow.conf 1_ROUTER _USERNAME my_userid 1_PASSWORD password 1_INTERFACE1 FastEthernet1/0/ _INTERFACE2 FastEthernet1/0/ _INTERFACE3 FastEthernet1/0/ _INTERFACE4 FastEthernet1/0/ _INTERFACE5 FastEthernet1/0/ _INTERFACE6 FastEthernet1/0/ # 2_ROUTER baku 2_USERNAME my_userid 2_PASSWORD password 2_INTERFACE1 FastEthernet1/0/ _INTERFACE2 FastEthernet1/0/ _INTERFACE3 FastEthernet1/0/ _INTERFACE4 FastEthernet1/0/ _INTERFACE5 FastEthernet1/0/ _INTERFACE6 FastEthernet1/0/

37 Setting up the routers l Get new IOS XE beta release IOS (tm) RSP Software (RSP-PV-M), Experimental Version 12.0( :193607) [rolsen-conn_isp_llq 144] Available at ftp://ftpeng.cisco.com/rolsen/rsp-pv-mz l Edge-Routers (Update of setup_flow required) class-map match-all EF match ip dscp 46 class-map match-all priority-qos match qos-group 99 policy-map emerge class EF bandwidth queue-limit 500 class priority-qos priority policy-map set-precedence –Attach service-policy set-precendence to every input –Attach service-policy emerge to every output interface

38 Setting up the routers l Interior-Routers (Update of setup_flow required) class-map match-all EF match ip dscp 46 policy-map emerge class EF bandwidth queue-limit 500 –Attach service-policy emerge to every output interface

39 Updating setup_flow l setup_flow is the script which is configuring the router l Possible update of setup_flow (is) 197 expect_after default { close; continue } expect "name: " { send "$username\r" } 200 expect "word: " { send "$password\r" } l Might be (without tacacs+) 197 expect_after default { close; continue } expect ”word: " { send "$password\r” } 200 expect “#” { send “enable\r” } 201 expect "word: " { send "$password\r" }

40 Updating the MDS l End-to-end API searches for end-systems in MDS l Publish information for all end-systems into MDS dslnet1.cldif: dn: hn=dslnet1.mcs.anl.gov, ou=Mathematics and Computer Science Division, o=Argonne National Laboratory, o=Globus, c=US …. If the machine is already in MDS this is enough networkreservationcontact: dslnet1.mcs.anl.gov:30000:/C=US/O=Globus/O=Argonne National Laboratory/OU=MCS/CN=roy197 expect_after default { close; continue } –run grid-info-update l Tools will be available soon

41 The GARA Source Tree l Subdirectories represent different layers –end2end –gara (gara/java GUI) –lram –resource_manager –slot_manager –common –logging –simple_multi –(dsrt) l We are providing examples for every layer l docs directory contains lots of useful information

42 Doing Further Tests l Select your favorite application l Determine port numbers l edit (set endpoint systems) and run script: gara/gara/utilities/demo_make_reservation l Create congestion l Try demo_modify_reservation for getting different premium bandwidth l Try demo_cancel_reservation l Or use the GUI

43 Timeline l Establish connectivity (January) –mostly done! –Setup accounts for Emerge-User (January!) –Check connectivity l Install and customize GARA (Jan - Feb) –partly done –Configure and check with GUI without any application (Just look at differv_manager and router) l Do basic experiments (February-March) l Start with multi-domain issues (March-April) l Port real applications to the Testbed (Feb-?)

44 Questions... l Now or later… mcs.anl.gov)