Overview of IETF work on IP traffic flow measurement and current developments Dr. Jürgen Quittek General Manager Network Research Division, NEC Europe Ltd. Heidelberg, Germany ITU-T Workshop on IP Traffic Flow Measurement (Geneva, Switzerland, 24 March 2011) Geneva, 24 March 2011

…… Flows can be long lasting... … or have a limited lifetime... … … … and packets may belong to more than one flow Typical reported flow information: start time end time #packets #bytes t Periodically reported for long lasting flows IP packets and flows Groups of IP packets sharing common characteristics (e.g IP src/dst address, TOS field, protocol, transport layer ports, etc.) 2

Classification & Flow Recording PAYLOAD HEAD Packet Capturing Filtering Sampling packets Filtering Sampling flow records packets flow records packet reports both steps may be trivial (1:1 sampling, no filtering) The general (passive) IP traffic measurement process Exporting process Observation Point (router, probe, etc.) Metering process 3

… … … … Meter: Filters packets, timestamps them and associates Pkts to flow(s) Flow cache: Creates/Removes/Updates flow records Flow Key Flow start time Flow last update time # Pkts # Bytes …. Collector: Receives export packets, interfaces to applications info Exp HD Database Exporter: Reads Flow cache, prepares and sends export packets info Exp HD Router functionality or dedicated Probe The flow monitoring process 4 IETF IPFIX (Netflow v9)

Flow monitoring issues Flows have very different characteristics long-/short-lived, high/low volume, etc. Creating/updating flow record at high speed links packet sampling fast memory for flow cache, flow sampling Timing out flows ( TCP FIN/RST vs. timeout ) Reporting flow cache reading effort, reporting frequency selective report Reporting format fixed format: Netflow 5 template based: Netflow 9, IPFIX 5

IETF activities on IP traffic measurement Three working groups IPPM: IP Performance Metrics defines metrics for performance measure- ments (delay, roundtrip time, loss, etc.) IPFIX: IP Flow Information eXport defines protocol for export of flow data PSAMP: Packet Sampling (concluded) defines protocol for export of packet data based on IPFIX 6

IPFIX protocol IP Flow Information eXport Established 2001 Main goal: Develop common IP traffic flow reporting protocol to be available on most future routers meeting requirements of many applications low hardware/software costs simple, Scalable extensible 7

Distinguishing flows by 5-tuple (IP addresses, protocol, port) MPLS label, TOS fields interface & direction Flexible aggregation of flows Metering Process timestamps flow timeouts Further requirements for IPFIX I 8

Extensible information/data model flow properties and statistics many header fields anonymization Reliable and secure data transfer congestion awareness push model reporting Configuration Further requirements for IPFIX II 9

IPFIX architecture Application Flow Record Observation Point Flow Information Export PAYLOAD HEAD Metering Process Exporting Process Collecting Process 10

O M E Probe O M E Simple Router OOOO M E Complex Router OOOO M OOOO M E Multiple Exporters OOOO M E OOO O M E Protocol Converter (Meter MIB) O M E O M E O M E ME Concen- trator CE Proxy C … IPFIX devices 11 C E M O Metering Process Exporting Process Collecting Process Observation Point

IPFIX protocol design Based on NetFlow version 9 Binary-coded flow record arrays Templates for flow record formats first send a template then send data records with the format defined by the template Runs over SCTP, TCP, UDP 12

IPFIX information model A flow record contains header fields (transport, IP, sub-IP) "flow keys" used for distinguishing flows counters for packets, bytes, etc. time stamps further flow properties min/max values, duration, direction next hop IP address BGP source AS, destination AS, next hop AS may also be used as flow keys All defined as "Information Elements" 13

IPFIX normative documents RFC 5101: Specification of the IPFIX Protocol for the Exchange of IP Traffic Flow Information, 2008 RFC 5102: Information Model for IPFIX, 2008 RFC 5103: Bidirectional Flow Export Using IPFIX, 2008 RFC 5473: Reducing Redundancy in IPFIX and PSAMP Reports, 2009 RFC 5610: Exporting Type Information for IPFIX Information Elements, 2009 RFC 5655: Specification of the IPFIX File Format, 2009 RFC 5815: Definitions of Managed Objects for IPFIX, 2010 core protocol specification 14

IPFIX informational documents RFC 3917: Requirements for IPFIX, 2004 RFC 3955: Evaluation of Candidate Protocols for IPFIX, 2004 RFC 5153: IPFIX Implementation Guidelines, 2008 RFC 5470: Architecture for IPFIX, 2009 RFC 5471: Guidelines for IPFIX Testing, 2009 RFC 5472: IPFIX Applicability, 2009 RFC 5982: IPFIX Mediation: Problem Statement,

Current issues in the IPFIX WG Configuration interface for configuring IPFIX devices defined as YANG module Mediation particularly for large networks driven by NTT aggregation anonymization Flow selection Structuring flow records extending IPFIX capabilities Using IPFIX for reporting other information MIB variables, SIP server logs, etc. 16

PSAMP Established in Summer 2002 Focus on sampling and capturing packets and on transferring them to data collectors Target applications traffic profiling monitoring network behavior Extends IPFIX export Defines packet sampling with much more detail packet filtering and sampling information model 17

IPPM "The IPPM WG will produce documents that define specific metrics and procedures for accurately measuring and documenting these metrics:" connectivity one-way delay and loss round-trip delay and loss delay variation loss patterns packet reordering bulk transport capacity (BTC = data_sent / elapsed_time) link bandwidth capacity Refer to WG official page for list of already published RFCs and ID 18

Final remarks The IETF developed IPFIX as standard protocol for reporting IP flow information Technology is mature many implementations several interoperability testing events major router vendors expected to release IPFIX soon as part of standard installation IPFIX is extensible BGP-related flow info can already be reported additional information elements can be added IPFIX can be used to report measurements at peering points appropriate metering hardware required 19