PART1 Data collection methodology and NM paradigms 1

Outline 2 Collection Infrastructure: How to Collect Data Records  Data collecting models  Network Design for the Collection Infrastructure  Communication Concepts  Collection Server Concepts  Placing the Collection Server

Collection Infrastructure: How to Collect Data Records This section describes the infrastructure required to collect accounting and performance data Infrastructure consists of metering devices collecting data records collection and mediation devices mediating data sets application servers generating business information 3

4

Data collecting models 5 Push Model Pull Model Event-Based Model

Push Model With the push model, records are not stored at the devices or are kept there for only a short time, until the device pushes them toward a collection server Therefore, when a relevant event occurs at the device, a notification is generated toward the management station 6

Push model An advantage of the push model is the event-driven aspect, which means that data is sent only if an event occurs. 7

Push model 8 For push model, the sending frequency depends on the technology as well. SNMP notifications are sent immediately upon occurrence of an event, while NetFlow uses a combination of triggers.

Pull model with the pull model, collection details are stored at the device until an external instance, such as a network management server, requests that they be sent. The pull model requires constant data retrieval, even if no event occurs at the device, because the device status can be identified only by polling. 9

10 The retrieval frequency of the pull model is configured at the management server; it determines how often data is retrieved from the network elements. Pull model

Pull model memory implementation The memory strategy at the network element for pull model can be implemented in three ways: In case of detailed accounting records, overwrite the oldest records when the maximum number of entries is reached. Wrap the counters when they reach the maximum value (for example, SNMP interface counters). Stop adding new entries before the existing ones have been retrieved or erased. 11

Event-Based Model The event-based model is the most effective combination of the push and pull model It builds on the advanced metering functionalities of the network element and combines them with the intelligence of the network management application 12

Event-based example By defining SNMP MIB thresholds for CPU utilization, interface utilization, link errors, and more, the network element monitors its status continuously. If a threshold is exceeded, a notification is pushed to the NMS application, which can start pulling for more details from the various MIB objects at the network element. After resolving the issue, the process returns to the initial state. 13

Network Design for the Collection Infrastructure how to transfer the collected data sets from the network elements to the collection server  One approach is to leverage the existing infrastructure that transports the users' traffic. This is also referred to as in-band management  An alternative is to set up a dedicated infrastructure for management purposes; this is also called the Data Communication Network (DCN). Related to networks, this concept is known as out-of-band (OOB) management. it uses either a dedicated network infrastructure (DCN) or a logical network on top of the shared infrastructure. 14

Communication Concepts The following communication concepts exist: Unicast (one to one) Multicast (one to many) Broadcast (one to all) Communication bus (a combination of unicast, multicast, and broadcast) 15

Unicast Unicast communication is applied in case of a central NMS server scenario, in which network elements communicate with a central server Because a single server is also a single point of failure, a backup server concept should be implemented. Although this increases availability, it also requires the device to send the records to both servers—either continuously or by checking the availability of the primary server and by sending traffic to the backup server only if the primary server is unavailable 16

Multicast An alternative to exporting data twice is to send it to a multicast address so that the export device does not need to inspect to which server the records need to be sent. 17

Broadcast broadcasting the messages to every device in the network is not recommended! 18

Communication bus All communication partners are connected to the bus and have a special listener component installed that monitors messages on the bus Messages are classified into specific categories and are broadcast on the bus To receive a message, you subscribe to one or multiple categories 19

Collection Server Concepts the collection server performs the following tasks: Data retrieval (pull or push) Monitoring the retrieved records and identifying lost data record loss between the device and the collection server (if the protocol supports it) Basic filtering functions (for example, filter all network management traffic) (optional) Threshold monitoring (optional) Data record formatting Data record storage 20

Placing the Collection Server (Centralized, Distributed) 21 A significant task is identifying how many collection servers are required and where in the network to place them.

22

23 You should calculate the estimated amount of generated accounting and performance data before selecting the central or distributed concept.

24 In a small or medium network, it can be sufficient to have one or two central servers and collect all accounting and performance records at these central instances Advantage: This eases the administration Disadvantage: Increases the network load, because all records are transferred from the devices to the servers

25 Best practice suggests a central design if Only a small number of (central) network elements collect data records The traffic overhead is low to medium. You want to monitor the traffic flows through the core of your network, it is sufficient to export the data records from the core devices to a central server.

26 distributing servers in the network reduces the transmitted traffic, because it gets sent after processing.

27 In case of usage-based billing, distributed collectors at the main remote locations are probably a better solution than deploying a central server.