Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed Chaudhry Usman Ali UNB, Fredericton

TESTING SIMULATOR ACCURACY Measure real network traffic. Compare real network traffic with the simulator results. Real TrafficSimulated Traffic Types of Traffic: CBR (Constant Bit Rate) data traffic. (Simplicity) FTP (File Transfer Protocol) session. (Popularity and Dynamic behaviour) CBR (Few external factors influence the model) FTP(Complex) HTTP(too many degrees of freedom)

METHODS OF SIMULATION Discrete Event Produce predictions in the network at a low level (packet- by-packet). Accurate. But SLOW ! Analytical Simulation Use mathematical models to produce their results. Sacrifice Accuracy ! Faster speed ! NS-2 and OPNET use a hybrid approach.

SIMULATION TOOLS OPNET MODELLER NS-2 1.Finite State Machine + Analytical Model special purpose modelling functions. 3.Not open source. 4.Model parameters can be altered. 1.Numerous models of Internet protocols. 2.NAM network animator. 3.Animations and graphs. 4.Custom applications and protocols can be created. C++ : per packet processing. Otcl : Simulation Scenario generation.

NETWORK TESTBED 5 PCs. Client-Server Pair. G- Traffic Generator. S - Traffic Sink. Hub 10/100 Mbit links. * KURT G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp , 2003.

NETWORK REPRESENTATION OPNETNS-2 G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp , 2003.

EXPERIMENTS CBR TrafficFTP Session 1.Characterized by a fixed bandwidth across the network. 2.Typically used by applications such as video and audio. 3.CBR traffic stream can be generated by fixing the packet size and using the same inter- arrival time between packets. Share, transfer and transmit information between two computers. Two types of test were performed for the FTP experiments. 1- FTP sessions using default settings. 2- Finely tuned FTP parameters.

PARAMETER TUNING Reno to New Reno Fast Recovery capability + recover without retransmission. Window Scaling Allows advertisement of window sizes bigger than 65 kB. Timestamp Imitates the echoing capability of the testbed in both directions. Link Latency Affects RTT. Max Segment Size Bigger Ethernet packets. Window Size More range for sliding the window value. G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet- Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp , 2003.

TRAFFIC SCENARIOS IP Traffic monitored using tcpdump. G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp , 2003.

RESULTS

Scenario#Client –Server LoadTraffic G. – Traffic S. Load CBR12 Mb/s0 Mb/s NS-2 displays a more realistic CBR behaviour. ROUTER`s PERSPECTIVE G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp , 2003.

Scenario#Client –Server LoadTraffic G. – Traffic S. Load CBR22 Mb/s CBR35 Mb/s0 Mb/s Lower throughput for the network testbed. Scheduling problems? ROUTER`s PERSPECTIVE G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp , 2003.

Scenario#Client –Server LoadTraffic G. – Traffic S. Load CBR45 Mb/s6 Mb/s CLIENT`s PERSPECTIVE Sudden drop ROUTER`s PERSPECTIVE SERVER`s PERSPECTIVE G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp , 2003.

Scenario#Client –Server LoadTraffic G. – Traffic S. Load FTP110 MB File0 Mb/s (CBR) FTP310 MB File0 Mb/s (CBR) Without tuning. With tuning. ROUTER`s PERSPECTIVE Server to Client Client to Server G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp , 2003.

Scenario#Client –Server LoadTraffic G. – Traffic S. Load FTP210 MB File6 Mb/s (CBR) FTP410 MB File6 Mb/s (CBR) Without tuning. With tuning. ROUTER`s PERSPECTIVE Server to Client Client to Server G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp , 2003.

Conclusions Pure CBR type traffic. ( NS-2 performed better). In one scenario OPNET performed “ better ”. FTP Experiments NS-2 FTP simulation model only indicated general transfer rate rather than replicating the actual network flow. OPNET performed closely to the testbed results. Simulation speed was fast in both simulators. The learning curve was different and steep at times. NS-2 : “ script” generated. OPNET : User Interface.

Conclusions Contd.. In FTP session, simulators did not model the dynamic behaviour of FTP in standard form. FTP adapts its output to prevailing network conditions. NS-2 and OPNET did not always mimic this performance. However, OPNET performed better.

Future Work Establishing a scheme to model HTTP in the simulators and the testbed.

REFERENCE G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp , 2003.