SWAN – A Simulator for Wireless “Ad Hoc” Networks L. Felipe Perrone <perrone@bucknell.edu> Dept. of Computer Science Bucknell University, Lewisburg, PA, U.S.A. 9 Ottobre 2006 Università di Bologna

What is Simulation? System input output If System is complex and defies mathematical analysis and/or is hard to control and to observe in experimental studies, one is better off constructing a model of the system and using it in a simulation study. Simulation is technique that replicates the behavior of a system using a model that approximates reality. It allows one to do experimental studies with the model and learn more about the system. 9 Ottobre 2006 Università di Bologna

Example: Simulating a Bank customers with transactions pending customers with transactions completed Imagine that the management wants to optimize the operation of the bank. Say that they want to make the customers’ experience better and minimize their costs. Observe the real system and create an abstraction that describes how things happen: customers arrive unpredictably, they enter a queue to perform a transaction, the teller serves them, and they leave the system. 9 Ottobre 2006 Università di Bologna

Experimenting with the Bank Model Simulating these two models, we can answer questions without much mathematical analysis: - Which bank model serves more customers per unit of time? - What is the average time that a customer spends in a bank? l 2m Bank Model m l m Bank Model 9 Ottobre 2006 Università di Bologna

Wireless Ad-Hoc Networks Battery operated computing devices. Communication happens via radio or laser beams. Nodes are free to move. Single or multihop. Automatic configuration. No need for infrastructure. 9 Ottobre 2006 Università di Bologna

Who Needs Simulation? We have a complex system that defies mathematical analysis. The system has several components tightly inter-connected. In order to validate and verify the correctness of the system, it will be necessary to run a good number of experiments. We wish to be able to have tight control over experimental conditions and to be able to reproduce them. 9 Ottobre 2006 Università di Bologna

Wish List for an WAN Simulator Accuracy Detail Completeness Performance Scalability 9 Ottobre 2006 Università di Bologna

Structure of a Wireless Ad Hoc Network Model (macro view) Environment Sub-models XDIM Space: geometry, terrain Mobility: single model, mixed models Propagation: computational simplicity (performance), accuracy (validity) YDIM 9 Ottobre 2006 Università di Bologna

Geometry Models Simple Torus 9 Ottobre 2006 Università di Bologna

Mobility Models Random Waypoint Model (individual) Choose a random destination (x,y). Choose a random speed U[min,max]. Mobile goes to (x,y). When mobile arrives at (x,y), it pauses for time p. Repeat. PS: Mobile moves in steps of time t. 9 Ottobre 2006 Università di Bologna

Mobility Models Group Choose a random destination (x,y) for the group center. All mobiles take one step toward (x,y). Repeat. PS: Mobile moves in steps of time t. 9 Ottobre 2006 Università di Bologna

Mobility Models Trace (x1,y1) Each mobile follows a predermined trajectory defined when the simulation first starts. M1 = [(x1,y1), (x2,y2), (x3,y3), ...] (x3,y3) (x2,y2) PS: Mobile moves in steps of time t. 9 Ottobre 2006 Università di Bologna

Propagation Models Friis Free-Space: No antenna height. Single wave front. Assumes line of sight is always possible. (Deterministic) TX RX Two-Ray Ground Reflection: Antenna heights considered. Assumes one wave front follows line of sight and another bounces off the ground. (Deterministic) TX RX Shadowing: Adds a stochastic component to Friis Free-Space. (Stochastic) TX RX 9 Ottobre 2006 Università di Bologna

Energy Model We implement the recommendations of the model developed by Laura Marie Feeney for IEEE 802.11b based on empirical measurements. There are different energy budgets for message transmission, message reception, and radio idle time. 9 Ottobre 2006 Università di Bologna

Structure of a Wireless Ad Hoc Network Model (micro view) heterogeneous or homogenous network Network Node Sub-models Physical Layer: radio sensing, bit transmission (SNRT, BER) MAC Layer: retransmissions, contention, throughput (IEEE 802.11) Network Layer: routing algorithms (AODV, DSR) Application Layer: traffic generation or “direct” execution of real application (CBR, VBR) APP APP APP NET NET NET MAC MAC MAC PHY PHY PHY RADIO PROPAGATION SUB-MODEL 9 Ottobre 2006 Università di Bologna

The Architecture of SWAN Physical Process read terrain features Protocol Graph Terrain Model OS Model (DaSSF Runtime Kernel) memory time run thread Host model read terrain features RF Channel Model 9 Ottobre 2006 Università di Bologna

The Scalable Simulation Framework (SSF) http://www.ssfnet.org Entity container for state variables SSF outChannel Process entity’s state evolution endpoints of communication links between entities inChannel Event messages between entities SSF is not a simulator: it’s a specification with bindings for Java and C++. 9 Ottobre 2006 Università di Bologna

SSF Modeling Entity A Entity B outChannel inChannel state state Event process inChannel outChannel process A.alignto(B) Timeline Channels have an associated delay which is used by the kernel to determine lookahead for parallel simulation. Channels are mapped to one another. Obviously large models would be painful to construct with this mechanism alone: enter DML (Domain Modeling Language). 9 Ottobre 2006 Università di Bologna

The DaSSF Implementation An SSF-compliant simulator requires: Fast threading mechanism, Efficient memory utilization, Portable: runs on sequential and parallel machines (shared and distributed memory); IRIX/Solaris/SunOS/Linux/OSF. DaSSF has evolved into iSSF and will see another incarnation in the near future. It was developed and is maintained by Jason Liu (Colorado School of Mines). 9 Ottobre 2006 Università di Bologna

DML and Design Patterns WIRELESS_NODE [ ID 1 xpos 0 ypos 0 battery 1000.0 graph [ ProtocolSession [ name "app" use "app.sensor-session" inter_arrival_time 0.002400 packet_size 100] name "net" use "net.aodv-session"] name "mac" use "mac.mac-802-11-session“] name "phy" use "phy.phy-802-11-session"] ] The model is described by a hierarchical list of key-attribute pairs. Each key is looked up in a database, a class is fetched, an object is constructed, and the list of attributes is passed to the corresponding constructor of the object. The model is constructed from the DML specification. 9 Ottobre 2006 Università di Bologna

SSFNET-like Architecture Our host descriptions were “borrowed” from the architecture of SSFNET. Protocol Graph A ProtocolSession models a protocol layer (as in the ISO/OSI reference model). A ProtocolGraph is a list of ProtocolSessions and models the complete protocol stack in a host. Adjacent ProtocolSessions communicate by exchanging ProtocolMessages. Application Session Transport Session Network Session Link Session PHY Session 9 Ottobre 2006 Università di Bologna

The ProtocolSession API ProtocolSession N+1 A ProtocolSession is a class that defines three methods: pop, push and control. An element higher in the stack can send it messages invoking push. An element lower in the stack can send it messages invoking pop. Anything that is not related to the protocol models is communicated using control. (pop) ProtocolSession N push control pop (push) ProtocolSession N-1 9 Ottobre 2006 Università di Bologna

ProtocolMessage PHY Header MAC Header Router+IP Header Data Message Network Application PHY Header MAC Header Router+IP Header Data Message A protocol message is a linked list of protocol headers. At each protocol session on the stack, the protocol header is added to the front that includes the control data of that protocol layer. When at the PHY layer, before the protocol message is sent out, a radio frame is constructed by packing all protocol headers into a bit stream. Radio frame is a derived class of SSF event. The data frame is sent to its destination. When received and accepted, the radio frame is unpacked and bubbles up the protocol stack. Each protocol session unwraps a protocol header off from the linked list. Radio Frame 9 Ottobre 2006 Università di Bologna

The development of SWAN Project started in 2000. First milestone: The simulation of 10,000 nodes running WiroKit, a proprietary routing algorithm developed by BBN Technologies. Second milestone: Used in the study of a high-performance model for IEEE 802.11b channel. Third milestone: Used as substrate in the development of a simulator for Berkeley motes running TinyOS. Fourth milestone: Used in an experimental study of lookahead enhancement techniques. Fifth milestone: Use in a study of best practices for modeling and simulation of wireless ad ho networks. Sixth milestone: Used in a study of attacks on wireless ad hoc networks. 9 Ottobre 2006 Università di Bologna

Validating and Verifying SWAN We looked for simulation studies done with other simulators that we could use as reference to validate SWAN. Roadblock: We found it very difficult to repeat previously published studies because we could not obtain information on all their settings (models and/or parameters). At times, we also failed to understand why certain parameter values had been chosen and perpetuated in the community. Roadblock: We could not find incontrovertible evidence that the simulators used in those studies had been validated. We resorted to comparing SWAN models to those of other simulators only to discover inconsistencies or errors in their models. 9 Ottobre 2006 Università di Bologna

Crisis, what crisis? Pawlikowski et al: “On credibility of simulation studies of telecommunication networks”. IEEE Communications Magazine 40 (1): “An opinion is spreading that one cannot rely on the majority of the published results on performance evaluation studies of telecommunication networks based on stochastic simulation, since they lack credibility. Indeed, the spread of this phenomenon is so wide that one can speak about a deep crisis of credibility.” 9 Ottobre 2006 Università di Bologna

Crisis indeed... Kotz et al. “The mistaken axioms of wireless-network research”. Technical Report TR2003-467, Dept. of Computer Science, Dartmouth College, July, 2003: “The ‘Flat Earth’ model of the world is surprisingly popular: all radios have circular range, have perfect coverage in that range, and travel on a two-dimensional plane. CMU's ns2 radio models are better but still fail to represent many aspects of realistic radio networks, including hills, obstacles, link asymmetries, and unpredictable fading. We briefly argue that key ``axioms'' of these types of propagation models lead to simulation results that do not adequately reflect real behavior of ad-hoc networks, and hence to network protocols that may not work well (or at all) in reality.” 9 Ottobre 2006 Università di Bologna

Why is it so difficult? Models for a wireless networks are complex and have many, many parameters. Articles in print can’t afford to list all the parameters used in a study. There isn’t a general consensus on the appropriate composition of the model (i.e. protocol stack) for wireless networks. We’re not all speaking the same language all the time: people may refer to the name of a well-known model and actually implement a different one (the terminology is sometimes perverted). Some of the people doing simulations lack wireless networking expertise (improper modeling), while others who have that expertise don’t understand much about simulation (improper output analysis). 9 Ottobre 2006 Università di Bologna

Exemple: Experimental Scenario RF propagation: 2-ray ground reflection, antenna height 1.5m, tx power 15dBm, SNR threshold packet reception. Mobility: density 7 neighbors per node, initial deployment triangular, stationary (pause=H, min=max=0), low (pause=60s, min=1, max=3), high (pause=0, min=1, max=10). Traffic generation: variation of CBR; session length=60s, ist=20s, destination is random for each session, CBR for each session, packet size=512 octets, vary packet rates to produce 16kbps, 56kbps, and 300kbps. Protocol stack: IEEE 802.11b PHY (message retraining modem capture), IEEE 802.11b MAC (DCF), ARP, IP, AODV routing. Arena size: variable; changed according to the number of nodes simulated to maintain constant density of 7 neighbors per node. Replications: 10 runs with different seeds for every random stream in the model. For all metrics estimated, we produced 95% confidence intervals. Scale: 20, 30, 40, and 50 nodes. 9 Ottobre 2006 Università di Bologna

Case Study: mobility model Yoon et al. “Random waypoint considered harmful”. INFOCOM 2003. Demonstrates how a bad choice of parameters can lead to a mobile network that tends to become stationary (no steady state). Called out attention to the fact that the vast majority of simulation studies with wireless networks ignores the ramp-up period in their sub-models. 9 Ottobre 2006 Università di Bologna

The impact of mobility transient on network metrics We verified that using data deletion to avoid the mobility transient led to significant changes in relative error: - from 5% to 30% in packet end-to-end delay, - from 5% to 30% in the ratio of data to control packets sent, - up to 10% in packet delivery ratio. Interesting results with algorithms for estimation of when steady-state is reached were presented at WSC ’03: Bause & Eickhoff. “Truncation Point Estimation Using Multiple Replications in Parallel”. PS: Our study shows that transients due to the ramp-up effect in traffic, further compromise the correctness of network metrics. (Perrone, Yuan, and Nicol. Best Practices in Modeling and Simulation of Wireless Networks. Winter Simulation Conference 2003). 9 Ottobre 2006 Università di Bologna

Lesson learned The simulation framework should be flexible enough in the collection of statistics to allow for data deletion. SWAN allows the modeler to define estimates for mobility and for network transient. All the statistics we collect are stored in data types derived from a base class that takes truncation point in time as a parameter. Only the values recorded after the truncation point are kept. In our experiments we run several simulations just to determine the right truncation point. It would be beneficial to automate this. Question: How many simulation replications does one need to produce reasonable estimates of 95% confidence of a given metric? 9 Ottobre 2006 Università di Bologna

Case study: composition of the protocol stack Broch et al. “A performance comparison of multi-hop wireless ad hoc networking protocols.” Mobicom ’98. States that the use of ARP in the protocol stack produces non-negligible effects in the simulation of a wireless network. We found no mention to the use of ARP models in other simulation studies save for one other paper. Our inquisitiveness lead us to attempt to quantify the effect of ARP on the networking metrics our simulation estimates. 9 Ottobre 2006 Università di Bologna

The impact of ARP For 16kbps and 56kbps traffic loads, the relative error in end-to-end delay observed was as high as 16%. Packet delivery ratio showed much less pronounced sensitivity: the relative error went only as high as 1.6%. The protocol contributes to the simulation with small processing load, and also with small additional memory requirement. 9 Ottobre 2006 Università di Bologna

Case study: radio interference model A common approach to reducing the complexity of interference computation is to limit, or truncate, the sensing range of a node. This range can be defined by a maximum path loss parameter. We have investigated two values: 106dB and 126dB. Results were consistent with what has been observed in the simulation of wireless cellular phone networks (Liljenstam & Ayani 1998; Perrone & Nicol 2000): - truncation leads to a substantial reduction in number of events to process at the cost of a small relative error in network metrics. For a given node, we can define a receiving range and a sensing range. 9 Ottobre 2006 Università di Bologna

A question of time How long does one need to run a simulation in order to produce good estimates of the network metrics? We have run simulations of 1000s after 500s of warm-up for mobility and traffic generation models. This choice, however, has proved to be insufficient to avoid problems… At high-traffic loads, due to contention and interference, the estimates obtained for end-to-end delay exhibit very large confidence intervals indicating that a higher number of samples should have been taken. 9 Ottobre 2006 Università di Bologna

Work for the future Sadly, the IEEE 802.11 model needs to be rewritten before we can distribute SWAN publicly. Need to allow simulations with mixed mobility models. Need to add support for improved radio propagation models. A tool for organizing and controlling experiments would be very helpful. 9 Ottobre 2006 Università di Bologna

SWAN Experiment Organizer Want to be able to create a template with pre-defined blanks for experimental parameters. Want to be able to create each particular experiment from combinations of experimental parameters. Want to be able to distribute experiments across a network of processors and to control their execution. Want to be able to have the simulation results be organized into a database that can be consulted from a web browser. 9 Ottobre 2006 Università di Bologna

SWAN Experiment Organizer WIRELESS_NODE [ ID 1 xpos 0 ypos 0 battery 1000.0 graph [ ProtocolSession [ name "app" use "app.tstapp-session" bitrate BITRATE packet_size PACKET on ON off OFF jitter JITTER] name "net" use "net.aodv-session"] name "mac" use "mac.mac-802-11-session“] name "phy" use "phy.phy-802-11-session"] ] PACKET=[128,512] BITRATE=[5,50,300] JITTER=[0,10,100] ON=[1,5,25,50] OFF=[54,50,30,5] 1TO1=[ON,OFF] [128,5,0,1,54] [128,5,0,5,50] [128,5,0,25,30] [128,5,0,50,5] ... [128,50,0,1,54] [512,5,0,1,54] 9 Ottobre 2006 Università di Bologna

Closing Words SWAN is maturing into a useful tool that has been applied in a number of different research studies. See http://www.eg.bucknell.edu/swan for a list of papers. Documentation is available, though it is at a very rough draft stage. It can be helpful to get one started. We have made strong efforts to document individual classes (Doxygen, DOC++). The tool is being prepared for public release. 9 Ottobre 2006 Università di Bologna

Acknowledgments The following people have participated in the SWAN Project: Jason Liu Yougu Yuan Noah Miller Evan Richardson Samuel Nelson Eric Graham Chris Kenna 9 Ottobre 2006 Università di Bologna