Chapter 2 : The ns-3 Network Simulator George F. Riley Thomas R. Henderson

ns-3 Introduction ns-3 (http://www.nsnam.org) is a free, open source software project building and maintaining a discrete-event network simulator for research and education Technical goals: Build and maintain a simulation core aligned with the needs of the research community Help to improve the technical rigor of network simulation practice

Research and education focus ns-3 themes Research and education focus Build and maintain simulation core, integrate models developed by other researchers Support research-driven workflows Open source development model Research community maintains the models Leverage available tools and models Write programs to work together Enforce core coding/testing standards

ns-3 is written in C++, with bindings available for Python ns-3 software overview ns-3 is written in C++, with bindings available for Python simulation programs are C++ executables or Python programs Python is often a glue language, in practice ns-3 is a GNU GPLv2-licensed project ns-3 is not backwards-compatible with ns-2

ns simulators: a brief history 1990 2000 2010 1988: REAL (Keshav) quarterly releases since ns-3.1 1990s: ns-1 1996: ns-2 1997-2000: DARPA VINT 2001-04: DARPA SAMAN, NSF CONSER 2006: ns-3 funded (NSF, INRIA) ns-3 features a new simulation core, with models drawn from GTNetS, ns-2, and the "yans" network simulator ns-3 core development (2006-08) The "ns" series of simulators originated with a simulator written by S. Keshav in the late 1980's, called REAL. In the early 1990's, Lawrence Berkeley National Laboratory (LBNL) developed ns-1, based on REAL, and primarily written in C++. In 1996, the DARPA VINT project began to fund a revision of ns, known as ns-2. ns-2 extended ns-1 by replacing the Tcl scripting language with MIT Object Tcl, and developing a new object framework. Funding for supporting ns-2 diminished by the end of 2004, and it remains lightly maintained. A new project, ns-3, was started in 2005. Also written in C++, the project decided to replace Tcl/OTcl scripting with Python bindings and to replace the simulator core to introduce more realism in the lower layers and to better support emulation modes. ns-3 made releases starting in June 2008 and as of January 2011, is working on its 11th release. June 2008: ns-3.1 release

Acknowledgment of support

2.1 Introduction

Why ns-3? Despite huge success with ns-2, the ns-3 developers had project goals that motivated the development of a new tool 1) Improve the realism of the models 2) Software reuse 3) Improved debugging 4) Consider long-term software maintenance

If the simulator cannot be made to closely model a real system: 1) attention to realism Problem: Research often involves a mix of simulations and testbed or live experiments If the simulator cannot be made to closely model a real system: hard to compare results or validate the model hard to reuse software between the two domains ns-3 solution: model nodes more like a real computer support key interfaces such as sockets API and IP/device driver interface (in Linux) reuse of kernel and application code

1) attention to realism (example) An ns-3 Node is a husk of a computer to which applications, stacks, and NICs are added Application Application Application (operating systems)

2) software integration Problem: why reimplement models and tools for which open-source implementations abound? ns-3 solution: ns-3 conforms to standard input/output formats so that other tools can be reused. e.g., pcap trace output, ns-2 mobility scripts ns-3 is adding support for running implementation code Network Simulation Cradle (Jansen) integration has met with success: Linux TCP code ns-3 “direct code execution” environment

2) software reuse (cont.) Example: ns-3 trace viewed with Wireshark:

3) Improved debugging Avoid split-language object implementations that proved to be hard to debug in ns-2 (OTcl/C++) ns-3 uses a C++ core with Python API bindings Layer a "helper" API on top of a powerful but complex low-layer API Different APIs for different user expertise Reduce memory errors via use of smart pointers

4) Software maintenance Open source projects are challenged to maintain the software over time Developers contribute models, then graduate Maintainers change over time ns-3 chose to use a more rigorous coding standard, detailed code reviews, and regression testing infrastructure ns-3 project decided not to devote scarce maintenance resources to develop and maintain ns-2 backward compatibility

2.2 Modeling the Network Elements in ns-3

2.2 Modeling the Network Elements in ns-3 Key objects in ns-3: nodes, devices, channels, protocols, packets, headers Application Application Application Application Packet Protocol stack Protocol stack Channel Node Node NetDevice NetDevice NetDevice

Random Variables Currently implemented distributions Uniform: values uniformly distributed in an interval Constant: value is always the same (not really random) Sequential: return a sequential list of predefined values Exponential: exponential distribution (poisson process) Normal (gaussian) Log-normal pareto, weibull, triangular, …

ns-3 random number generator Uses the MRG32k3a generator from Pierre L'Ecuyer http://www.iro.umontreal.ca/~lecuyer/myftp/papers/streams00.pdf Period of PRNG is 3.1x10^57 Partitions a pseudo-random number generator into uncorrelated streams and substreams Each RandomVariable gets its own stream This stream partitioned into substreams Independent replications are handled by incrementing the run number to obtain uncorrelated substreams

Tracing and statistics Tracing is a structured form of simulation output Example (from ns-2): + 1.84375 0 2 cbr 210 ------- 0 0.0 3.1 225 610 - 1.84375 0 2 cbr 210 ------- 0 0.0 3.1 225 610 r 1.84471 2 1 cbr 210 ------- 1 3.0 1.0 195 600 r 1.84566 2 0 ack 40 ------- 2 3.2 0.1 82 602 + 1.84566 0 2 tcp 1000 ------- 2 0.1 3.2 102 611 Problem: Tracing needs vary widely would like to change tracing output without editing the core would like to support multiple outputs

ns-3 has a new tracing model ns-3 solution: decouple trace sources from trace sinks Benefit: Customizable trace sinks Trace source Trace source Trace sink Trace source configurable by user unchanging

ns-3 tracing various trace sources (e.g., packet receptions, state machine transitions) are plumbed through the system Documented in a central place

Basic tracing Helper classes hide the tracing details from the user, for simple trace types ascii or pcap traces of devices std::ofstream ascii; ascii.open ("wns3-helper.tr"); CsmaHelper::EnableAsciiAll (ascii); CsmaHelper::EnablePcapAll ("wns3-helper"); YansWifiPhyHelper::EnablePcapAll ("wsn3-helper");

Multiple levels of tracing Highest-level: Use built-in trace sources and sinks and hook a trace file to them Mid-level: Customize trace source/sink behavior using the tracing namespace Low-level: Add trace sources to the tracing namespace Or expose trace source explicitly

Highest-level of tracing Use built-in trace sources/sinks, and hook a trace file to them // Also configure some tcpdump traces; each interface will be traced // The output files will be named // simple-point-to-point.pcap-<nodeId>-<interfaceId> // and can be read by the "tcpdump -r" command (use "-tt" option to // display timestamps correctly)‏ PcapTrace pcaptrace ("simple-point-to-point.pcap"); pcaptrace.TraceAllIp ();

Mid-level of tracing Mid-level: Customize trace source/sink behaviour with tracing namespace void PcapTrace::TraceAllIp (void)‏ { NodeList::Connect ("/nodes/*/ipv4/(tx|rx)", MakeCallback (&PcapTrace::LogIp, this)); } Regular expression editing Hook in a different trace sink

ns-3 attribute system Problem: Researchers want to identify all of the values affecting the results of their simulations and configure them easily ns-3 solution: Each ns-3 object has a set of attributes: A name, help text A type An initial value Control all simulation parameters for static objects Dump and read them all in configuration files Visualize them in a GUI Makes it easy to verify the parameters of a simulation

Use cases for attributes An Attribute represents a value in our system An Attribute can be connected to an underlying variable or function e.g. TcpSocket::m_cwnd; or a trace source

Use cases for attributes (cont.)‏ What would users like to do? Know what are all the attributes that affect the simulation at run time Set a default initial value for a variable Set or get the current value of a variable Initialize the value of a variable when a constructor is called The attribute system is a unified way of handling these functions

How to handle attributes The traditional C++ way: export attributes as part of a class's public API walk pointer chains (and iterators, when needed) to find what you need use static variables for defaults The attribute system provides a more convenient API to the user to do these things

Navigating the attributes Attributes are exported into a string-based namespace, with filesystem-like paths namespace supports regular expressions This allows individual instances of attributes to be manipulated Attributes also can be referenced by name without specifying a path through the object namespace e.g. “ns3::WifiPhy::TxGain” This allows users to manipulate a global (default) value of the attribute A Config class allows users to manipulate the attributes

Attribute namespace strings are used to describe paths through the namespace Config::Set ("/NodeList/1/$ns3::Ns3NscStack<linux2.6.26>/net.ipv4.tcp_sack", StringValue ("0"));

Navigating the attributes using paths Examples: Nodes with NodeIds 1, 3, 4, 5, 8, 9, 10, 11: “/NodeList/[3-5]|[8-11]|1” UdpL4Protocol object instance aggregated to matching nodes: “/$ns3::UdpL4Protocol”

What users will do Attribute Value e.g.: Set a default initial value for a variable Config::Set (“ns3::WifiPhy::TxGain”, DoubleValue (1.0)); Syntax also supports string values: Config::Set (“WifiPhy::TxGain”, StringValue (“1.0”)); Attribute Value

Fine-grained attribute handling Set or get the current value of a variable Here, one needs the path in the namespace to the right instance of the object Config::SetAttribute(“/NodeList/5/DeviceList/3/Phy/TxGain”, DoubleValue(1.0)); DoubleValue d; nodePtr->GetAttribute ( “/NodeList/5/NetDevice/3/Phy/TxGain”, v); Users can get pointers to instances also, and pointers to trace sources, in the same way

ns-3 attribute system Object attributes are organized and documented in the Doxygen Enables the construction of graphical configuration tools:

Attribute documentation

Options to manipulate attributes Individual object attributes often derive from default values Setting the default value will affect all subsequently created objects Ability to configure attributes on a per-object basis Set the default value of an attribute from the command-line: CommandLine cmd; cmd.Parse (argc, argv); Set the default value of an attribute with NS_ATTRIBUTE_DEFAULT Set the default value of an attribute in C++: Config::SetDefault ("ns3::Ipv4L3Protocol::CalcChecksum", BooleanValue (true)); Set an attribute directly on a specic object: Ptr<CsmaChannel> csmaChannel = ...; csmaChannel->SetAttribute ("DataRate", StringValue ("5Mbps"));

2.3 Simulating a Computer Network in ns-3

Four basic steps to perform Create the network topology Create the data demand on the network Execute the simulation Analyze the results Program listing 2-1 annotates the file "first.cc", which is the first tutorial program discussed in the ns-3 tutorial

2.4: Smart Pointers in ns-3

Smart Pointers Memory management in a C++ program is a complex process, and is often done incorrectly or inconsistently. We have settled on a reference counting design using so-called "smart-pointers" All objects maintain an internal reference count that allows the object to safely delete itself when the count goes to zero We provide a class called "Ptr" that is similar to the intrusive_ptr of the Boost C++ library This implementation allows you to manipulate the smart pointer as if it was a normal pointer: you can compare it with zero, compare it against other pointers, assign zero to it, etc.

2.5: Representing Packets in ns-3

2.5: Representing Packets in ns-3 The design of the Packet framework of ns-3 was heavily guided by a few important use-cases: avoid changing the core of the simulator to introduce new types of packet headers or trailers maximize the ease of integration with real-world code and systems make it easy to support fragmentation, defragmentation, and, concatenation which are important, especially in wireless systems. make memory management of this object efficient allow actual application data or dummy application bytes for emulated applications Each network packet contains a byte buffer, a set of byte tags, a set of packet tags, and metadata.

Packet class organization each Packet has a reference to a Buffer holding packed packet data class Packet provides the user API Packet Tags contain simulation-specific information such as cross-layer messages, or simulation flow IDs optional PacketMetadata provides metadata about the contents of the buffer to enable (e.g.) pretty-printing

Copy-on-write semantics Packets implement copy-on-write semantics to reduce the number of deep data buffer copies during a simulation Packets are reference-counted objects; multiple clients can hold references to the same packet, and the packet is automatically freed when all references are deleted When more than one reference to a packet buffer exists, the packet buffer can be shared unless a so-called "dirty"operation (such as editing a protocol header) is performed by one of the packet users

2.6: Object Aggregation in ns-3

ns-3 object aggregation Problem: coupling between models hinders software reuse in different configurations must intrusively edit the base class for this or, leads to C++ downcasting, e.g.: // Channels use Node pointers, but here I really want a // MobileNode pointer, to access the MobileNode API double WirelessChannel::get_pdelay(Node* tnode, Node* rnode) { // Scheduler &s = Scheduler::instance(); MobileNode* tmnode = (MobileNode*)tnode; MobileNode* rmnode = (MobileNode*)rnode; known as the C++ “weak base class” problem

ns-3 object aggregation (cont.) ns-3 solution: an object aggregation model objects can be aggregated to other objects at run-time a “query interface” is provided to ask whether an particular object is aggregated similar in spirit to COM or Bonobo objects // aggregate an optional mobility object to a node: node->AggregateObject (mobility); ... // later, other users of node can query for the optional object: senderMobility = i->first->GetNode ()->GetObject<MobilityModel> (); // we did not have to edit class Node (base class), or downcast!

2.7: Events in ns-3

Events in discrete-event simulation Simulation time moves discretely from event to event C++ functions schedule events to occur at specific simulation times A simulation scheduler orders the event execution Simulation::Run() gets it all started Simulation stops at specific time or when events end

Events in ns-3 An event in ns-3 is simply a function pointer with some state (the technical term is a "functor") Any object method or static function can be used as an event handler in ns-3 e.g. Simulator::Schedule (function name, object pointer, arguments); This is different from several other discrete-event simulators, where special Handler() methods provide a centralized place for processing events on an object

2.7: Events in ns-3 An event: At time 10 seconds, static void ExampleFunction (MyModel *model) { std::cout << "ExampleFunction received event at " << Simulator::Now ().GetSeconds () << "s" << std::endl; model->Start (); } int main (int argc, char *argv[]) MyModel model; Simulator::Schedule (Seconds (10.0), &ExampleFunction, &model); Simulator::Run (); Simulator::Destroy (); An event: At time 10 seconds, call ExampleFunction() with the argument &model.

2.8 Compiling and Running the Simulation

ns-3 uses the waf build system Waf is a Python-based framework for configuring, compiling and installing applications. It is a replacement for other tools such as Autotools, Scons, CMake or Ant http://code.google.com/p/waf/ For those familiar with autotools: Autotools -> Waf equivalent configure -> ./waf -d [optimized|debug] configure make -> ./waf

Running programs Programs are built as build/<variant>/path/program-name programs link shared library libns3.so Using ./waf --shell ./waf --shell ./build/debug/samples/main-simulator Using ./waf --run ./waf --run examples/csma-bridge ./waf --pyrun examples/csma-bridge.py

2.9: Animating the Simulation

Animation Overview ns-3 not directly funding visualizers and configurators no “official” tool; expect multiple to be developed Not integrated (directly) with ns-3 Ns-3 creates “animation” file, visualizers use this as input and create the animation. netanim, pyviz, and nam for ns-3

Pyviz: A Python-based animator

NetVis

NetVis / XML Interface <anim lp = "0”> <topology minX = "1000" minY = "2382.3" maxX = "10900" maxY = "8617.7"> <node lp = "0" id = "0" locX = "4000" locY = "5500” image = "Satellite.png" imageScale = "10.0"/> <node lp = "0" id = "1" locX = "7000" locY = "5500” image ="Satellite.png" imageScale = "5.0"/> <link fromLp = "0" fromId = "0" toLp = "0" toId = "1"/> </topology> <packet fromLp = "0" fromId = "11" fbTx = "0.66483" lbTx = "0.665264"> <rx toLp = "0" toId = "1" fbRx = "0.66583" lbRx = "0.666264"/> </packet> <wpacket fromLp = "0" fromId = "49" fbTx = "0.549245" lbTx = "0.549497” range = "250"> <rx toLp = "0" toId = "16" fbRx = "0.549497" lbRx = "0.549749"/> <rx toLp = "0" toId = "18" fbRx = "0.549497" lbRx = "0.549749"/> <rx toLp = "0" toId = "29" fbRx = "0.549497" lbRx = "0.549749"/> <rx toLp = "0" toId = "32" fbRx = "0.549498" lbRx = "0.549749"/> <rx toLp = "0" toId = "36" fbRx = "0.549498" lbRx = "0.549749"/> <rx toLp = "0" toId = "37" fbRx = "0.549497" lbRx = "0.549749"/> </wpacket> </anim>

2.10: Scalability with Distributed Simulation

Overview Parallel and distributed discrete event simulation Allows single simulation program to run on multiple interconnected processors Reduced execution time! Larger topologies! Terminology Logical process (LP) Rank or system id -- PDES, probably most are familiar with it (PDNS) -- Allows single simulation program to run on multiple interconnected processors -- Basically, take simulation topology, split it up in to chunks, run in on different processors -- Reduce execution time, larger topologies -- IMPORTANT, dist sim must have same results as sequential sim -- Term: LP – each sequential simulation termed LP. In general, each LP is mapped to an individual processor -- Term: Rank – way of identifying all of the logical processes, unique integer, system id. Nodes are explicitly assigned ranks.

Figure 1. Simple point-to-point topology Quick and Easy Example -- simple dumbbell topology you might see in a tutorial. -- 10 nodes, imagine scaling it up factor of 10 or 100, and lots of data -- could take a while -- why not distributed it? -- most obviously, split it right down the middle and simply have 2 LPs Figure 1. Simple point-to-point topology

Figure 2. Simple point-to-point topology, distributed Quick and Easy Example -- nodes on the left are one LP (rank 0), nodes on the right another LP (rank 1) -- each LP could be executed by a different processors -- how does this work though? -- imagine packet originating on left side, destined for right side -- what is involved to make sure this all goes smoothly? Figure 2. Simple point-to-point topology, distributed

Implementation Details LP communication Message Passing Interface (MPI) standard Send/Receive time-stamped messages MpiInterface in ns-3 Synchronization Conservative algorithm using lookahead DistributedSimulator in ns-3 -- first, the LPs must communicate. Think about last example, rank0 must communicate with rank 1 in some way -- for this we use MPI, popular for many parallel and distributed applications -- essentially allows LPs to communicate by sending and receiving time-stamped messages -- created an mpi-interface in ns-3 to easily use this standard -- another issue is what is called synchronization. -- describe synch. Issues -- different types of algorithms, conservative / optimistic -- conservative algorithm, lookahead -- distributedsimulator in ns-3, takes care of synch., and determines what events can be executed

Implementation Details (cont.) Assigning rank Currently handled manually in simulation script Next step, MpiHelper for easier node/rank mapping Remote point-to-point links Created automatically between nodes with different ranks through point-to-point helper Packet sent across using MpiInterface -- nodes must be assigned rank so we can determine which LP to run them on. -- currently this is done manually when you create the nodes in your simulation script -- plan on adding an MpiHelper so that you can feed in a configuration file with a node – rank mapping -- this way it is much easier to modify existing simulations for distributed use -- another implementation detail is remote point-to-point links -- currently only support dividing the simulation along p2p links -- imagine dumbbell example, when packet comes to p2p link, serialized, sent via MPI to rank 1 -- these p2p links are created automatically with point-to-point-helper

Implementation Details (cont.) Distributing the topology All nodes created on all LPs, regardless of rank Applications are only installed on LPs with target node -- final implementation detail, how the topology is distributed -- all nodes created on all LPs, regardless of rank -- only apps are installed on LPs with target node -- simplifies routing -- people have raised some concerns about this methodology -- what is wanted is nodes to only be created on certain ranks -- some routing tricks are needed to support this method (simulator A doesn’t know about nodes on simulator B) -- pdns does this by terminating remote p2p links and assigning it an IP address -- remote links with matching network addresses are logically connected -- could take an approach like this in the future Figure 3. Mixed topology, distributed

Performance Test DARPA NMS campus network simulation Allows creation of very large topologies Any number of campus networks are created and connected together Different campus networks can be placed on different LPs Tested with 2 CNs, 4 CNs, and 6 CNs -- with implementation details out of the way, move on to some performance testing -- DARPA nms campus network simulation -- read slide (will show a picture) -- also tested with a greater number of CNs, but left that out here

Campus Network Topology -- describe pic Figure 4. Single campus network

2 Campus Networks Figure 5. Execution time with 2 campus networks -- describe graphs -- mention that these runs weren’t done rigourously -- other people were using the machines -- just to show the general performance Figure 5. Execution time with 2 campus networks Figure 6. Speedup with 2 LPs

Summary Distributed simulation in ns-3 allows a user to run a single simulation in parallel on multiple processors By assigning a different rank to nodes and connecting these nodes with point-to-point links, simulator boundaries are created Simulator boundaries divide LPs, and each LP can be executed by a different processor Distributed simulation in ns-3 offers solid performance gains in time of execution for large topologies -- basically read the slides

2.11: Emulation Capabilities

Emulation support Support moving between simulation and testbeds or live systems A real-time scheduler, and support for two modes of emulation GlobalValue::Bind (“SimulatorImplementationType”, StringValue (“ns3::RealTimeSimulatorImpl”));

Various hybrids of the above are possible ns-3 emulation modes real machine ns-3 Testbed 2) testbeds interconnect ns-3 stacks real machine virtual machine virtual machine ns-3 1) ns-3 interconnects real or virtual machines Various hybrids of the above are possible

Example: ns-3 with Linux Network Namespaces Linux (FC 12 or Ubuntu 9.10) machine ns-3 Wifi ghost node ghost node Container Container TapBridge WiFi tapX tapY /dev/tunX /dev/tunY Makes use of a "ghost node" that bridges packets to a Linux namespace container running the protocol stack and applications

Support for use of Rutgers WINLAB ORBIT radio grid Example: ORBIT and ns-3 Support for use of Rutgers WINLAB ORBIT radio grid Image from ORBIT - Wireless Network Testbed website

Issues in Performing "Co-Simulation" Ease of use Configuration management and coherence Information coordination (two sets of state) e.g. IP/MAC address coordination Output data exists in two domains Debugging Error-free operation (avoidance of misuse) Synchronization, information sharing, exception handling Checkpoints for execution bring-up Inoperative commands within an execution domain Deal with run-time errors Soft performance degradation (CPU) and time discontinuities

2.12: Analyzing the Results

NSF CISE Community Research Infrastructure Frameworks for ns-3 NSF CISE Community Research Infrastructure University of Washington (Tom Henderson), Georgia Tech (George Riley), Bucknell Univ. (Felipe Perrone) Project timeline: 2010-14 Figure courtesy of Dr. Felipe Perrone, Bucknell University

Automation Inspired by SWAN-Tools and ANSWER frameworks. User interfaces, description languages, and tools for automation of experiments. Model composition, structural validation, control of experiments, data processing and storage, and archiving experimental setup.

Topology generation Integrate BRITE topology generator (Boston Univ.) into framework. BRITE is downloaded into distribution and compiled by the ns-3 build system. The ns-3 simulation script uses a topology helper which reads a BRITE configuration file, receives results from BRITE, and builds the ns-3 topology.

Higher-level modeling and experiment description Provide a model and a description of experiment. Framework generates design of experiment space, distribute simulation runs to machines, collect results and archive in persistent storage. User mines storage to find, extract, and visualize results. Figure courtesy of Dr. Felipe Perrone, Bucknell University

General architecture Figure courtesy of Dr. Felipe Perrone, Bucknell University