Interconnect simulation

Different levels for Evaluating an architecture Numerical models – Mathematic formulations to obtain performance characteristics (close form or non-close form) Simulation – Simulating the hardware in a program Emulation Prototype Operational system

Simulation *Jim Kurose, University of Massachusets, Amherst system under study (has deterministic rules governing its behavior) exogenous inputs to system (the environment) system boundary observer “real” life computer program simulates deterministic rules governing behavior psuedo random inputs to system (models environment) program boundary observer “simulated” life

Why simulation? real-system not available, is complex/costly or dangerous (eg: space simulations, flight simulations) quickly evaluate design alternatives (eg: different system configurations) evaluate complex functions for which closed form formulas or numerical techniques not available

Simulation advantages and limitations advantages: – sometimes cheaper – find bugs (in design) in advance – generality: over analytic/numerical techniques – detail: can simulate system details at arbitrary level drawbacks: – caution: does model reflect reality? – large scale systems: lots of resources to simulate (especially accurately simulate) – may be slow (computationally expensive – 1 min real time could be hours of simulated time) – art: determining right level of model complexity – statistical uncertainty in results

Types of simulators Time driven simulation – Cycle/instructor based simulation – Stop and observe the system after a fixed time interval (cycle, instruction, etc) – Events are assumed to happen at the time interval boundary Event driven simulation – Move from one event to another – Not at a fixed interval – Always simulate the first event in the future – this event may trigger new events before the second event.

Types of simulators For network simulation, it is more common to use event driven simulation technique. – Flow-level simulation (fluid model) How each flow is transmitted Topology and routing issues – Packet-level simulation How each packet is sent from the source to the destination More detailed packet level issues such as packet scheduling, congestion control, etc. – Flit-level simulation How each flit goes through the network E.g. examine the performance of wormhole routing

Programming an event driven interconnect simulator Define a set of events – E.g. Message arrival, message departure, etc Each event will trigger the change of the state in the system The simulator simulates the system activities for the events – Change system states E.g. when a message starts communicating, the throughput of other messages may be affected. – Adding future events if necessary.

Programming an event-driven simulator An event driven simulator must maintain the following: – simulated time: internal (to simulation program) variable that keeps track of simulated time – system “state”: variables maintained by simulation program define system “state” e.g., may track number (possibly order) of packets in queue, current value of retransmission timer – events: points in time when system changes state each event has associate event time – e.g., arrival of packet to queue, departure from queue – precisely at these points in time that simulation must take action (change state and may cause new future events) model for time between events (probabilistic) caused by external environment

Event driven simulator block diagram initialize event list get next (nearest future) event from event list time = event time update statistics done? n process event (change state values, add/delete future events from event list)

An example: a flow level network simulator Input: – a list of messages as well as their arrival time src_id, dst_id, msg_size, arrival_time Output – the finishing time for each message Assumption: The topology and the routing scheme is given. – Topology: Link between switches and its bandwidth – Routing: given a pair of src_id and dst_id, routing givens thelist of links along a path from source to destination.

An example flow level simulator Events – Message arrival and message departure Modeling network activities – Given a set of active flows, we must model the transmission rates for each active flow. Need to make assumptions One possible assumption: the flow rate is equal to the equal share of the link bandwidth for the most loaded link along its path. Another possible assumption: MAX-MIN fairness.

An example flow level simulator Assumption 1: the flow rate is equal to the equal share of the link bandwidth for the most loaded link along its path. Assumption 2: MAX-MIN fairness Active flows: 1->5, 2->3, 3->5, 4->5, link bandwidth 1Mbps – Assumption 1: 1->5, 3->5, 4->5 333Mbps, 2->3, 500Mbps – Assumption 2: 1->5, 3->5, 4->5 333Mbps, 2->3, 667Mbps

An example flow level simulator Network state: – Active messages, sizes of reminding active messages, rate for each active message, may be the finishing time for each active message based on the current data rate. – When a new message arrives, the number of active messages increases: the network state needs to be recomputed – When a message finishes, the number of active message decreases: the network state needs to be recomputed.

An example flow level simulator initialize event list get next (nearest future) event from event list time = event time update statistics done? n process event (change state values, add/delete future events from event list) put all messages in priority queue get next (nearest future) (first arrival time or finishing time) time = event time If message finish, output time/msg done? n Change the set of active flows Recompute the reminding message size, new rate, and new finishing time

An example flow level simulation Will write up a homework and prepare topology and routing files. How accurate if the simulator? – A lot of assumptions – this is common for all simulation type of work. – If the assumption or the model is off, simulation produces junk results.

Another example: a packet level simulator Objective: to investigate different packet scheduling algorithms used in routers. Input: – a list of messages as well as their arrival time src_id, dst_id, msg_size, arrival_time – The messages can be generated following some distribution – to evaluate the performance under different traffic conditions. Output – the finishing time for each message

An example packet level simulator Events: – Message arrives a compute node, packet at a compute node, packet arrives at a port, packet moves to an output port, etc. Model system activities – Modeling the compute node behavior – Model router behavior

An example packet level simulator Model system activities – Modeling the compute node behavior When a message arrives, packetize the message, insert packets into a host packet queue (with ready time) to be injected to the network. When a packet is received, if it is the last packet of a message, keep the statistics.

An example packet level simulator – Modeling router behavior Each input port can be modeled by an input queue Each output port can be modeled by an output queue Packet scheduling algorithm: when multiple packets are ready at the input queue, the algorithm determines which packet gets to be scheduled to the output queue. Modeling router latency: after a packet is removed from an input queue, insert the packet to the output queue with a time stamp equal to current_time + latency – Modeling link latency?

An example packet level simulator Network states: – Queues on routers and end hosts, message at end hosts The simulator basically figures out the next future event, and perform the event – A message arrives at end host: packetize the message – A packet is ready on end host: move the packet – A packet is ready on a queue: move the packet

An example packet level simulator initialize event list get next (nearest future) event from event list time = event time update statistics done? n process event (change state values, add/delete future events from event list)

Packet level.vs. flow level simulation Flow level simulation has higher level of abstraction – more details are lost. Is flow level simulation always more efficient?

Network simulation packages For network simulation, the driver is quite standard. – Difference in the modeling techniques Quite some network simulators are available: – NS2, OpNet, Mininet, etc – Provide many common library funcationality – Allow build the network topology and routing schemes. When to use the existing simulators and when to write one from scratch?