IPFIX Aggregation draft-dressler-ipfix-aggregation-01.txt

Motivation Reduction of monitoring data Speed-up of flow accounting Bandwidth savings and performance savings at the collector Speed-up of flow accounting Reduction of concurrent active streams in a monitor Concentrating multiple IPFIX streams Definition of concentrator functionality Transport of information about the aggregation rules For improved processing of IPFIX data 63rd IETF Meeting, Paris, 2005

Architecture EP EP EP AP AP MP MP MP CP CP exported monitoring data (IPFIX Protocol) exported monitoring data (IPFIX Protocol) EP EP EP AP AP MP MP MP CP CP exported monitoring data (IPFIX Protocol) EP: Exporting Process AP: Aggregation Process MP: Metering Process 63rd IETF Meeting, Paris, 2005

Aggregation Rules Specify Comprise aggregation instructions containing which flow records to aggregate into a meta-flow record how the meta-flow record and the corresponding data template looks like Comprise aggregation instructions containing IPFIX field ID mandatory field for incoming records included in meta-flow record or data template depending on field modifier pattern (optional) restricts aggregated flow records to those that match this pattern field modifier (discard, keep, mask/n, or aggregate) specifies how this field is treated implicitly defines if the field appears in meta-flow or data template 63rd IETF Meeting, Paris, 2005

Field Modifiers Rule instruction Result Field modifier Pattern exist Field in meta-flow record contains Fixed-value field in Data Template contains discard no n/a yes pattern keep original value original value, if pattern is range of values mask/n IP network address 63rd IETF Meeting, Paris, 2005

Field Modifier – cont’d Special field modifier aggregate for counters, timestamps etc. Result depends on field: minimum in case of minimumPacketLength, minimumTtl, flowStartSeconds, flowStartMilliSeconds maximum in case of maximumPacketLenth, maximumTtl, flowEndSeconds, flowEndMilliSeconds binary OR (as suggested by IPFIX-INFO) in case of ipv6OptionHeaders, tcpControlBits sum in case of octetDeltaCount, packetDeltaCount 63rd IETF Meeting, Paris, 2005

Example Goal: Aggregation Rule: monitor flows to web servers (http/https) in 10.10.0.0/16 aggregate sources addresses into /24 network addresses Aggregation Rule: discard protocolIdentifier discard sourceTransportPort mask/24 sourceIpv4Address discard destinationTransportPort in 80,443 keep destinationIpv4Address in 10.10.0.0/16 aggregate packetDeltaCount aggregate octetDeltaCount aggregate flowStartMilliSeconds aggregate flowEndMilliSeconds 63rd IETF Meeting, Paris, 2005

Example – cont’d Data Template: 63rd IETF Meeting, Paris, 2005 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID | Field Count = 6 | | Data Count = 2 | Preceding Rule | | Field 1 Type = sourceIpv4SourceNetwork | | Field 2 Type = destinationIpv4Address | | Field 3 Type = packetDeltaCount | | Field 4 Type = octetDeltaCount | | Field 5 Type = flowStartMilliSeconds | | Field 6 Type = flowEndMilliSecondsess | | Data 1 Type = destinationTransportPort | | Data 1 Value = 80,443 | | Data 2 Type = destinationIpv4Network | | Data 2 Value = 10.10.0.0/16 | 63rd IETF Meeting, Paris, 2005

Example – cont’d Incoming flows: Resulting meta-flow: pattern in data template Incoming flows: Resulting meta-flow: Prot Src Port Src Addr Dst Port Dst Addr # Pkt # Oct Start End TCP 64235 10.0.1.1 80 10.10.0.10 4 144 1055 1090 64236 3 56 1071 1103 6889 10.0.1.2 2 34 1083 1100 5555 10.0.2.1 6 155 1201 6666 10.10.0.11 77 1095 1199 discarded fixed-value in data template Src Net Dst Addr # Pkt # Oct Start End 10.0.1.0/24 10.10.0.10 9 234 1055 1103 10.0.2.0/24 6 155 1090 1201 10.10.0.11 3 77 1095 1199 63rd IETF Meeting, Paris, 2005

Cascading Aggregation Rules Goal: Allows other semantics than “match-any”, i.e. may be used to avoid that an incoming flow contributes to more than one meta-flow Cascading aggregation rules: Use preceding rule field in data template header Get incoming flow preceding rule Apply rule 1? no preceding rule Apply rule 2? no yes Aggregate … yes Aggregate 63rd IETF Meeting, Paris, 2005

Conclusions IPFIX Aggregation -00 received only positive feedback -01 has reached a good state Already two implementations supporting aggregation IBM Erlangen University / Tuebingen University Next steps To be continued as an individual I-D? To be added to the IPFIX charter? 63rd IETF Meeting, Paris, 2005