Draft-molina-flow-selection-00 Maurizio Molina,

2 © NEC Europe Ltd., 2002 Network Laboratories, Heidelberg Motivation, Background (1/2) Flow selection (i.e. meter and/or export only a subset of the flows) may be needed for –Reducing on purpose exporter load & traffic –Resource limitation In Flow recording process: cannot store all flow recs In Meter: update Flow rec is lightweight; create new Flow rec is heavyweight  limit creation rate In exporter: not able to export all the Flow recs Resource limitation may become more evident under attacks

3 © NEC Europe Ltd., 2002 Network Laboratories, Heidelberg Motivation, Background (2/2) Flow selection is considered in current IPFIX drafts, limited to the exporting process –Arch draft (10.1) selection criteria of flows for export Attempt here: –Consider also meter & flow recording processes –Define which information to export about flow selection –Why? Fundamental for collectors to adjust their level of trust of received information Presentation was given in Wien –

4 © NEC Europe Ltd., 2002 Network Laboratories, Heidelberg Metering Process Packets may be discarded because they would lead to the creation of a new Flow rec –and there’s no room in the Flow recording process –or the Flow rec creation rate must be kept low There are “intelligent” methods for doing so [*][**] (not in the scope of standardization) What info to export? –(A) Macro-flow of these discarded packets (#pkts, #bytes, timestamp first, timestamp last) NOTE: it’s a discarding that happens after flow classification! [*] C. Estan and G. Varghese: New Directions in Traffic Measurement and Accounting, ACM SIGCOMM Internet Measurement Workshop 2001, San Francisco (CA) Nov [**] M.Molina: A scalable and efficient methodology for flow monitoring in the internet, International Teletraffic Congress (ITC-18), Berlin, Sep. 2003

5 © NEC Europe Ltd., 2002 Network Laboratories, Heidelberg Flow recording process Existing Flow recs may be discarded without being exported to make room for new ones –Looks strange? Yes, but [*] proofs that can be a better choice! What info to export? –(B1) Macro-flow of the non-exported packets and bytes belonging to these discarded records (#pkts, #bytes, timestamp first, timestamp last) –(B2) Macro flow of these discarding events (#Frec, timestamp first, timestamp last) –In addition, export (C) the amount of all non-exported traffic contained in the flow recording process (#pkts, #bytes, #Frecs containing at least 1 non-exported pk). –Why? (next slide will clarify…) –No timestamps, in this case, would require keeping full state…. [*] M.Molina: A scalable and efficient methodology for flow monitoring in the internet, International Teletraffic Congress (ITC-18), Berlin, Sep. 2003

6 © NEC Europe Ltd., 2002 Network Laboratories, Heidelberg Exporting process (1/2) Flow recs may not be exported –For explicit policies (e.g. export 1 flow out of N, or only the flows with more than x bytes, or other smarter strategies, e.g. [*]) –For resource limitation (buffer for packet preparation, exporting rate control…) Non exported records may be –immediately discarded –kept in the flow recording process for later exporting [*] N. Duffield, C. Lund, M.Thorup: Learn More, Sample Less: Control of Volume and Variance in Network Measurement -

7 © NEC Europe Ltd., 2002 Network Laboratories, Heidelberg Exporting process (2/2) What info to export? –(D1) Macro-flow of the non-exported packets and bytes belonging to non exported and discarded records (#pkts, #bytes, timestamp first, timestamp last) –(D2) Macro flow of these discarding events (#Frec, timestamp first, timestamp last) (D1)  (B1), (D2)  (B2) only the discarding cause is different! –(E) the amount of all non-exported traffic contained in the flow recording process that was not exported, but that still stays in the flow repository (#pkts, #bytes, #Frecs containing at least 1 non-exported pk) –No timestamps, in this case, would require full state…. (E) may be difficult/impossible to obtain depending on the flow recording reading implementation –That’s the reason of having (C): At a collector (C)+(B)+(D)-received traffic= (E)

8 © NEC Europe Ltd., 2002 Network Laboratories, Heidelberg Open Issues Is the defined info about flow selection sufficient/redundant? We didn’t define so far info about the flow selection process, like –“Exporter selects 1 out of N flows” or “Exporter exports flows with more than x bytes only” or “meter does elephant flow selection” Is it feasible/worth to describe it? How to export flow selection information? –It is not relative to a flow, but to the behavior of a whole metering / flow recording / exporting process –As protocol draft suggests, “The Options Template Record (and its corresponding Options Data Record) is used to supply information about the Metering Process configuration or Metering Process specific data, rather than supplying information about IP Flows” –Option recs contain a “scope field” which is “The relevant portion of the Exporting Process/Metering Process to which the Options Template Record refers. Currently defined values are: can be the interface, the cache, etc.” –Are the currently defined scope types (System, Interface, Line Card, Cache, Template) enough for flow sampling purposes?