1. Proposed Diffserv Evaluation for Emerge Sites Volker Sander sander@mcs.anl.gov Argonne National Laboratory 1/20/2000

2. 2 Outline l Basic GARA overview l New GARA-features l Basic set of experiments l Installation of GARA l Running the experiments using GARA l Timeline

3. 3 The Basic Diffserv Concept Aggregate based scheduling implements EF PHB Classify and Mark packets Shape Traffic Flow based Classify and Mark packets Shape Traffic

4. 4 Proposed Implementation Client GARA API GARA Diffserv Resource Manager Server

5. 5 The Multidomain Extension Server GARA Diffserv Resource Manager Client GARA API GARA Diffserv Resource Manager

6. 6 GARA Globus Architecture for Reservation and Allocation l Three important contributions: –Support for Advance Reservations (AG) –Uniform treatment of underlying resources >Network, disk, CPU etc… >Currently supported: Differentiated Services, DPSS, and DSRT –Layered architecture enables generic co- reservation agents >User is able to select API best suitable for his need

7. 7 GARA Basics—Reservations l There is a generic “reservation”, which has several properties: –Start Time (“now” or future) and Duration –Resource type/Underlying resource identifier –Resource-specific (bandwidth, % CPU…) l All reservations are treated uniformly: –Create/Modify (Given properties) => Returns Reservation Handle –Destroy –Monitor (Callbacks or Polling)

8. 8 A picture of what happens (creating a reservation) Gatekeeper (Authenticate and Authorize) Create User Program or Agent Reservation Handle Resource Manager Gatekeeper Service Local Resource Manager API (LRAM) Resource (Router...)

9. 9 A picture of what happens (claiming a reservation) User Program bind(handle) LRAM API Resource Manager Setup reservation Resource Enforce Reservation Note: GARA supports third party binding

10. 10

11. 11 Example: Network reservation l In advance, make a reservation:  RSL: & (reservation-type=network) (start-time=now) (duration=3600) (endpoint-a=128.135.11.4) (endpoint-b=128.135.11.1) (bandwidth=1000) (protocol=udp) (reservation-subtype=low-latency)  Handle: Opaque string H 1 l At run-time: –Bind(H 1,RSL)  RSL: & (which-endpoint=a) (endpoint-a-port=5000) (endpoint-b-port=5000)

12. 12 The Resource Manager l To track advanced reservations, we use a simple “slot table” l We assume exclusive access to resource through the resource manager l We aren’t tied to this particular resource manager % of available resource allocated 0% 100% Time

13. 13 End-to-End Network Reservations l Algorithm for a single network reservation –Discover which gatekeepers/resource managers need to be contacted (MDS) –Make reservation at each one. User process Workstation Resource Manager MDSEnd-to-End Co-Reservation Agent

14. 14 The GARA API l globus_gara_reservation_create()  Gatekeeper contact  RSL reservation specification  Reservation Handle or Error l globus_gara_reservation_modify()  Old Reservation Handle  RSL reservation specification  New Reservation Handle or Error

15. 15 The GARA API (continued) l globus_gara_reservation_cancel()  Reservation Handle  Error l globus_gara_reservation_status()  Reservation Handle  Status/Error l globus_gara_reservation_callback_register()  Reservation Handle  Callback Function/User Parameter  Error

16. 16 New GARA features l New Reservation-Subtypes –low-latency >Used to support jitter sensitive applications using the same EF aggregate behavior as high-throughput TCP applications >Based on traffic shaping and Priority-Queuing >Currently being evaluated –background >Used to support bulk transfers, including deadline support. Takes at least the amount of premium traffic required to fulfill the deadline; more if not used actively l Monitoring of the network edges –Provides feedback to applications when they send too fast

17. 17 Network + CPU Reservations

18. 18 Example of bulk transfer (LAN)

19. 19 Example of bulk transfer (WAN)

20. 20 Feedback from Monitoring the Edges

21. 21 Evaluation Tools l UDP Traffic Generator –Modified Version of Andy Adamson’s gen_send and gen_recv >Evaluate admission control >Creating competing traffic –MGEN/Drec >Evaluate delay and Jitter for Premium UDP Flow l Modified Version of ttcp –GARA-enabled (wait for reservation) –Support for a desired application rate >long term control >short term control –Consecutive bandwidth reporting –Bulk transfer ttcp

22. 22 Basic Experiments l Evaluate Admission Control l Evaluate premium TCP Flows under Congestion –Analyze network traces with tcpdump l Evaluate UDP –Delay –Jitter l Evaluate Inter-Domain Issues Fundamental idea: Get the necessary knowledge to be able to experiment with REAL applications

23. 23 Basic Experiment I l Goal: Proof of Admission Control –Gather packet statistics from gen_recv for different configurations and different traffic profiles:

24. 24 Basic Experiment I gen_send GARA Diffserv Resource Manager gen_recv

25. 25 Basic Experiment II l Goal: Proof of Premium Quality for TCP flows –Run modified ttcp with different traffic and reservation profiles; gather receiver bound statistics –Create congestion using gen_send/recv

26. 26 Basic Experiment II l Enhancement –Use tcpdump and tcptrace get a deep overview on the behavior –Use netlogger_enabled ttcp_gara and gen_recv to publish data into MDS. “Real Time Visualization”

27. 27 Basic Experiment II GARA Diffserv Resource Manager ttcp -t ttcp -r gen_sendgen_recv

28. 28 Basic Experiment III l Goal: Demonstrate Premium Quality for UDP flows –Run mgen/drec with different traffic and reservation profiles; gather receiver bound statistics –Create congestion using gen_send/recv

29. 29 Basic Experiment III GARA Diffserv Resource Manager mgen drec gen_sendgen_recv

30. 30 Basic Experiment IV l Goal: Proof Premium Quality for UDP flows sharing one EF aggregate –Run mgen/drec with different traffic and reservation profiles; gather receiver bound statistics –Run high-throughput ttcp using a correct reservation –Create congestion using gen_send/recv

31. 31 Basic Experiment IV GARA Diffserv Resource Manager ttcp -t ttcp -r mgendrecgen_recvgen_send

32. 32 Additional Experiments l Bulk-Transfer l Reevaluate inter-domain behavior –Add Traffic-Shaping between domains l Test Inter-Domain Admission control l Prepare the Instrumentation l Go with real applications

33. 33 Installing GARA l Get GARA –(currently through E-Mail from sander/roy @mcs.anl.gov) l Get SSLeay 0.9.0 and install it l Get Globus 1.1 configure and compile it deploy is not necessary for doing tests but recommended Run grid-cert-request to apply for a Grid X509 Certificate add “gara-service stderr_log,local_cred - $GARA_DIR/gara/programs/globus_gatekeeper_gara_service -d” to $GLOBUS_DEPLOY_DIR/etc/globus-services l Read gara/docs/* l Edit gara/build l Run gara/build l Customize GARA –gara/resource_manager/programs/diffserv_manager.conf –gara/resource_manager/programs/setup_flow.cfg –(Samples are in the documentation directory)

34. 34 Building GARA l $GARA_DIR/build l #!/bin/ksh l CC=cc l CXX=CC l CFLAGS="-g -v -mt" l GLOBUS_DIR=/soft/pub/packages/globus-1.1.0/development/sparc-sun-solaris2.7_pt l hreads_standard_debug l SSL_LIBRARY_DIR=/soft/pub/packages/SSLeay-0.9.0/lib l export CC CXX CFLAGS GLOBUS_DIR SSL_LIBRARY_DIR l configure l make clean l make depend l make all l # l # For portability reasons you might not want to do a make withdsrt.. l # in case: just comment out this line. l #make withdsrt l Run build

35. 35 Configuring GARA l resource_manager/programs/diffserv_manager.conf PublicationMethod web mds LDAPDistinguishedName "dslnet1.mcs.anl.gov:34880:/C=US/O=Globus/O=Argonne Nati onal Laboratory/OU=MCS/CN=roy” MDSPublishPassword “password” WEBPublishFile /home/sander/public_html/diffserv_manager_slots.html NoOfRouters 2 IPAddressesServed[1] 140.221.48.162,140.221.48.146,140.221.48.164,140.221.48.1 65,140.221.48.166,140.221.48.167,140.221.48.34 IPAddressesServed[2] 140.221.48.98,140.221.48.114,140.221.48.102,140.221.48.10 3,140.221.48.104,140.221.48.105 l Parameters: "LDAPDistinguishedName", "MDSPublishPassword", "WEBPublishFile", "FILEPublishFile", "Quantity", "ClearSlotTable", "SlotTableFilename", "Verbose", "PublicationMethod", "DontReallyBindReservations"

36. 36 Configuring GARA l resource_manager/programs/setup_flow.conf 1_ROUTER 140.221.48.145 1_USERNAME my_userid 1_PASSWORD password 1_INTERFACE1 FastEthernet1/0/1 140.221.48.162 1_INTERFACE2 FastEthernet1/0/1 140.221.48.164 1_INTERFACE3 FastEthernet1/0/1 140.221.48.165 1_INTERFACE4 FastEthernet1/0/1 140.221.48.166 1_INTERFACE5 FastEthernet1/0/1 140.221.48.167 1_INTERFACE6 FastEthernet1/0/0 140.221.48.146 # 2_ROUTER baku 2_USERNAME my_userid 2_PASSWORD password 2_INTERFACE1 FastEthernet1/0/1 140.221.48.98 2_INTERFACE2 FastEthernet1/0/1 140.221.48.102 2_INTERFACE3 FastEthernet1/0/1 140.221.48.103 2_INTERFACE4 FastEthernet1/0/1 140.221.48.104 2_INTERFACE5 FastEthernet1/0/1 140.221.48.105 2_INTERFACE6 FastEthernet1/0/0 140.221.48.114

37. 37 Setting up the routers l Get new IOS XE beta release IOS (tm) RSP Software (RSP-PV-M), Experimental Version 12.0(19991129:193607) [rolsen-conn_isp_llq 144] Available at ftp://ftpeng.cisco.com/rolsen/rsp-pv-mz.120699 l Edge-Routers (Update of setup_flow required) class-map match-all EF match ip dscp 46 class-map match-all priority-qos match qos-group 99 policy-map emerge class EF bandwidth 30000 queue-limit 500 class priority-qos priority policy-map set-precedence –Attach service-policy set-precendence to every input –Attach service-policy emerge to every output interface

38. 38 Setting up the routers l Interior-Routers (Update of setup_flow required) class-map match-all EF match ip dscp 46 policy-map emerge class EF bandwidth 30000 queue-limit 500 –Attach service-policy emerge to every output interface

39. 39 Updating setup_flow l setup_flow is the script which is configuring the router l Possible update of setup_flow (is) 197 expect_after default { close; continue } 198 199 expect "name: " { send "$username\r" } 200 expect "word: " { send "$password\r" } l Might be (without tacacs+) 197 expect_after default { close; continue } 198 199 expect ”word: " { send "$password\r” } 200 expect “#” { send “enable\r” } 201 expect "word: " { send "$password\r" }

40. 40 Updating the MDS l End-to-end API searches for end-systems in MDS l Publish information for all end-systems into MDS dslnet1.cldif: --------------------- dn: hn=dslnet1.mcs.anl.gov, ou=Mathematics and Computer Science Division, o=Argonne National Laboratory, o=Globus, c=US …. If the machine is already in MDS this is enough networkreservationcontact: dslnet1.mcs.anl.gov:30000:/C=US/O=Globus/O=Argonne National Laboratory/OU=MCS/CN=roy197 expect_after default { close; continue } –run grid-info-update l Tools will be available soon

41. 41 The GARA Source Tree l Subdirectories represent different layers –end2end –gara (gara/java GUI) –lram –resource_manager –slot_manager –common –logging –simple_multi –(dsrt) l We are providing examples for every layer l docs directory contains lots of useful information

42. 42 Doing Further Tests l Select your favorite application l Determine port numbers l edit (set endpoint systems) and run script: gara/gara/utilities/demo_make_reservation l Create congestion l Try demo_modify_reservation for getting different premium bandwidth l Try demo_cancel_reservation l Or use the GUI

43. 43 Timeline l Establish connectivity (January) –mostly done! –Setup accounts for Emerge-User (January!) –Check connectivity l Install and customize GARA (Jan - Feb) –partly done –Configure and check with GUI without any application (Just look at differv_manager and router) l Do basic experiments (February-March) l Start with multi-domain issues (March-April) l Port real applications to the Testbed (Feb-?)

44. 44 Questions... l Now or later… (sander/roy @ mcs.anl.gov)