Efficient Network Management (236635) SNMP TCP Backchannel Submitted by: An SNMP agent extension for transferring large files Tsachi Sharfman Kfir Karmon

Efficient Network Management (236635) Problem Description SNMP runs over UDP SNMP runs over UDP UDP is unreliable and therefore is inefficient for transferring large amounts of data UDP is unreliable and therefore is inefficient for transferring large amounts of data

Efficient Network Management (236635) Solution ’ s Requirements Should enable the efficient transfer of any SNMP table supported by the agent. Should enable the efficient transfer of any SNMP table supported by the agent. Should be transparent to existing SNMP clients. Should be transparent to existing SNMP clients.

Efficient Network Management (236635) Possible Solutions (1/3) Download a mirrored table using FTP Download a mirrored table using FTP Based on widespread known technology (SNMP & FTP) Relies on an reliable protocol (FTP/TCP)  Not transparent to existing SNMP clients (Fails to withstand the requirements) 1.Send an SNMP SET command to a special OID 2.The agent will locally save a specified table 3.The client will retrieve the table using FTP Pros and Cons:

Efficient Network Management (236635) Possible Solutions (2/3) Use SNMP through a TCP tunnel Use SNMP through a TCP tunnel Full transparency for existing SNMP clients Relies on an reliable transfer protocol (TCP)  Non standard, relies on proprietary technology  Overhead for short SNMP requests (UDP is cheaper) 1.Open a TCP tunnel between the client and server 2.SNMP packets are routed through the TCP tunnel Pros and Cons:

Efficient Network Management (236635) Possible Solutions (3/3) Use a TCP backchannel for SNMP responses Use a TCP backchannel for SNMP responses Full transparency for existing SNMP clients Relies on an efficient transfer protocol (TCP)  Might be non-friendly to firewalls and NAT gateways 1.Client requests the table using SNMP 2.Client has a service, listening on a designated TCP port 3.Agent sends the SNMP responses encapsulated in the TCP backchannel to the client ’ s listening port 4.The service locally emulates the agent ’ s SNMP response Pros and Cons:

Efficient Network Management (236635) Implementing Solution #3 In order to trigger the Backchannel (BC) mechanism the client should : In order to trigger the Backchannel (BC) mechanism the client should : –Register its IP and listening port –Register the wanted table (OID) –Issue an SNMP request to the Mirrored OID At this point the SNMP agent will query itself with SNMP for the requested OID (the real one) At this point the SNMP agent will query itself with SNMP for the requested OID (the real one) After the receiving the table a bakchannel to the client is opened and the table is transferred After the receiving the table a bakchannel to the client is opened and the table is transferred The client side will send itself an SNMP response with the received table The client side will send itself an SNMP response with the received table Architecture Overview - TCP backchannel (1/1)

Efficient Network Management (236635) Implementing Solution #3 New MIB defined in the “ NET-SNMP-EXAMPLES-MIB.txt ” file New MIB defined in the “ NET-SNMP-EXAMPLES-MIB.txt ” file It introduced two table: It introduced two table: –BCRegisteredClientsTable Every client that wants to use the BC mechanism needs to register itself here –BCRegisteredOIDsTable Each OID that needs to be transferred via the BC should be registered here An additional “ phantom ” table was introduced, it includes OIDs that issuing SNMP requests to them triggers the BC mechanism An additional “ phantom ” table was introduced, it includes OIDs that issuing SNMP requests to them triggers the BC mechanism Backchannel Control MIB (BCM) 1/3

Efficient Network Management (236635) Implementing Solution #3 BCRegisteredClients BCRegisteredClients BCRegisteredOIDs BCRegisteredOIDs Legend: Backchannel Control MIB (BCM) 2/3

Efficient Network Management (236635) Implementing Solution #3 BCMirroredOIDsTable BCMirroredOIDsTable –These names are not defined in the MIB, though the structure is supported by the agent add-on (via the numeric annotation) Legend: Backchannel Control MIB (BCM) 3/3

Efficient Network Management (236635) Implementing Solution #3 The NET SNMP agent was extended using the, existing, add-on mechanism The NET SNMP agent was extended using the, existing, add-on mechanism The extension would handle any SNMP requests that are targeted to the “… BackchannelControl.* ” OIDs The extension would handle any SNMP requests that are targeted to the “… BackchannelControl.* ” OIDs When an SNMP Get* request, to the MirroredOIDs, is received the agent will query itself for the appropriate OID When an SNMP Get* request, to the MirroredOIDs, is received the agent will query itself for the appropriate OID The agent will send, via the TCP BC, the SNMP packets constructing the received response (in the TCP packets payloads) The agent will send, via the TCP BC, the SNMP packets constructing the received response (in the TCP packets payloads) Backchannel Server (BCServer) 1/1

Efficient Network Management (236635) Implementing Solution #3 A.K.A BackChannel Listener (BL) A.K.A BackChannel Listener (BL) The client registers itself and the wanted OIDs to the designated tables as described above. The client registers itself and the wanted OIDs to the designated tables as described above. The client issues a standard SNMP Request to the MirroredOID The client issues a standard SNMP Request to the MirroredOID The BCClient listens to the designated port for a connection to be made by the BCServer The BCClient listens to the designated port for a connection to be made by the BCServer After receiving the encapsulated SNMP response packets from the BCServer, the BCClient issues a standard SNMP response “ spoofed ” with the server ’ s IP. After receiving the encapsulated SNMP response packets from the BCServer, the BCClient issues a standard SNMP response “ spoofed ” with the server ’ s IP. Thus causing the initial, standard, SNMP request to receive the response without knowing that it was sent via a TCP BC. Thus causing the initial, standard, SNMP request to receive the response without knowing that it was sent via a TCP BC. Backchannel Client (BCClient) 1/1

Efficient Network Management (236635) Implementing Solution #3 The Whole Picture

Efficient Network Management (236635) How We Tested (1/2) I order to test the efficiency of the new BC mechanism we built the following network topology I order to test the efficiency of the new BC mechanism we built the following network topology Windows SNMP Client + BCClientWindows SNMP Agent + BCServer Linux based router + Configurable Packet loss driver

Efficient Network Management (236635) How We Tested (2/2) We created scripts to automate our testing: We created scripts to automate our testing: –_AddRoutingRules.vbs This script added routing rules to the agent ’ s routing table, a rule was added for each IP in the range: 10.0.[0-4].[0-255] –_Test.vbs This was the main script, its usage: _T est.vbs [TCP | UDP] [Server's IP] [Client's IP] [NumOfRepititions] [NumOfTimes] [CSV_File] This script performs snmpBulkGet requests to the server using a defined number of repetitions in the bulk response. This script performs snmpBulkGet requests to the server using a defined number of repetitions in the bulk response. The requests were executed a defined number of times. The requests were executed a defined number of times. The output was exported to a CSV formatted file for later data processing in MS-Excel. The output was exported to a CSV formatted file for later data processing in MS-Excel. TCP requests were issued to the mirrored OID and UDP requests were issued to the regular ipRoutingTable OID (both via standard SNMP) TCP requests were issued to the mirrored OID and UDP requests were issued to the regular ipRoutingTable OID (both via standard SNMP)

Efficient Network Management (236635) Results (1/4) At first we compared the BC mechanism ’ s efficiency against the standard SNMP with the following parameters: At first we compared the BC mechanism ’ s efficiency against the standard SNMP with the following parameters: –60 repetitions in the bulk response (= a single IP packet) –Packet Loss rate ranging from 0%-50% stepping in 10% –Each scenario was tested 50 times Increase Packet Loss Rate & no IP Fragments

Efficient Network Management (236635) Results (2/4) Then we used the following parameters: Then we used the following parameters: –126 repetitions in the bulk response (= 2 IP packet, caused by IP- fragmentation) –Packet Loss rate ranging from 0%-50% stepping in 10% –Each scenario was tested 50 times Increase Packet Loss Rate & 2 IP Fragments

Efficient Network Management (236635) Results (3/4) Then after we used the following parameters: Then after we used the following parameters: –400 repetitions in the bulk response (= ~8 IP packet, caused by IP- fragmentation) –Packet Loss rate ranging from 0%-50% stepping in 10% –Each scenario was tested 50 times Increase Packet Loss Rate & 8 IP Fragments

Efficient Network Management (236635) Results (4/4) The last test had the following parameters: The last test had the following parameters: –Repetitions ranging from 6 to 2880, in the bulk response, stepping in ~400 –Packet Loss rate of 10% –Each scenario was tested 50 times Increasing IP Fragments & fixed Packet Loss

Efficient Network Management (236635) Observations (1/2) Both methods success rate drop when packet loss rate inclines Both methods success rate drop when packet loss rate inclines Though, BC success rate is consistently higher than standard SNMP Though, BC success rate is consistently higher than standard SNMP UDP ’ s average success time is constant, BC ’ s average time increases. (UDP fails at const timeout, BC retries) UDP ’ s average success time is constant, BC ’ s average time increases. (UDP fails at const timeout, BC retries) BC ’ s performance declines when packet loss increases over 20% (Combination of average time and success rate) BC ’ s performance declines when packet loss increases over 20% (Combination of average time and success rate) Increase Packet Loss Rate

Efficient Network Management (236635) Observations (2/2) When packet loss rate set to 10% and increasing the size of the bulk... When packet loss rate set to 10% and increasing the size of the bulk... BC delivers great performance: Nearly fixed time & high success rate BC delivers great performance: Nearly fixed time & high success rate Standard SNMP suffers from increasing average time and steep drop in success rate Standard SNMP suffers from increasing average time and steep drop in success rate Fixed Low Packet Loss

Efficient Network Management (236635) Conclusion Using a TCP Backchannel proved to be very effective in relatively moderate packet loss rates (~10%) Using a TCP Backchannel proved to be very effective in relatively moderate packet loss rates (~10%) The solution is transparent to existing SNMP based management tools The solution is transparent to existing SNMP based management tools Provides efficient method for retrieving large amounts of data in a quick and reliable manner Provides efficient method for retrieving large amounts of data in a quick and reliable manner

Efficient Network Management (236635) The End