Licentiate Seminar: On Measurement and Analysis of Internet Backbone Traffic Wolfgang John Department of Computer Science and Engineering Chalmers University of Technology Göteborg, Sweden

Licentiate Seminar Wolfgang John Internet, 1983 Internet, 2005 Why measure Internet traffic? (1) The Internet is changing in size ARPANET, 1969

Licentiate Seminar Wolfgang John The Internet is changing in application Why measure Internet traffic? (2)

Licentiate Seminar Wolfgang John The Internet –is constantly developing –is used differently in different locations –is heterogeneous The Internet is not understood in its entirety! INTERconnected NETworks Why measure Internet traffic? (3) INTER NET

Licentiate Seminar Wolfgang John Operational purpose –Troubleshooting, provisioning, planning …. Scientific purpose –Protocols, infrastructure and services –Performance properties –Internet simulation models –Security measures Why measure Internet traffic? (4)

Licentiate Seminar Wolfgang John Thesis Objectives 1.Guidelines for Internet measurement 2.Current traffic characteristics 3.Traffic decomposition 4.Inconsistent behavior

Licentiate Seminar Wolfgang John Outline Measurement approaches Internet measurement challenges The MonNet project Scientific contribution Results –Four studies included Conclusions Measurement Analysis

Licentiate Seminar Wolfgang John Measurement approaches Network traffic measurement Active Passive Software Hardware Online Offline Flows Packets Complete Headers Different protocol levels Statistical summaries Transport layer

Licentiate Seminar Wolfgang John Internet measurement challenges (1) Legal considerations Ethical and moral considerations Operational considerations Technical considerations

Licentiate Seminar Wolfgang John Measurement challenges (3) Technical considerations Data amount –Exhausting I/O and storage access speeds Data reduction techniques –Filtering, sampling, packet truncation Timing –Clock synchronization

Licentiate Seminar Wolfgang John The MonNet Project (1) Technical Solution 10 Gbps Göteborg splitter Borås 10 Gbps Processing Platform and Storage Measurement Node 2 Measurement Node 1

Licentiate Seminar Wolfgang John The MonNet Project (2) Internet Regional ISPs Göteborg Stockholm Other smaller Univ. and Institutes Göteborgs Univ. Student- Net Chalmers Univ. Measurement location Borås April traces (20 minutes) 11 billion packets, 7.6 TB of data Sept. – Nov traces (10 minutes) 28 billion packets, 19.5 TB of data

Licentiate Seminar Wolfgang John Scientific Contribution Level of complexity Quantification of inconsistent behavior Traffic characterization Packet level Flow level Traffic classes Study I Study II Study IV Study III Upcoming

Licentiate Seminar Wolfgang John Study I: Packet Level Analysis Updated packet-level characteristics of Internet traffic Inconsistencies in headers will appear –Network attacks and malicious traffic –Active OS fingerprinting –Buggy applications or protocol stacks

Licentiate Seminar Wolfgang John High level analysis does not necessarily show differences → detailed analysis does! 2 main reasons for directional differences: –Malicious traffic the Internet is “unfriendly” –P2P Göteborg is a P2P source P2P is changing traffic characteristics e.g. packet sizes, TCP termination, TCP option usage Study II: Flow level analysis

Licentiate Seminar Wolfgang John Study III: Classification Method (1) Classification of flow traffic without payload Heuristics to identify nature of endpoints Rules based on connection patterns and port numbers –5 rules for P2P traffic –10 rules to classify other types of traffic remove ‘false positives’ from P2P

Licentiate Seminar Wolfgang John Study III: Classification Method (2) # connections in 10 6 Amount of data in TB Comparison of classification methods for P2P traffic

Licentiate Seminar Wolfgang John Study III: Classification Method (3) Previous classification methods on packet header traces don’t work well on backbone data Proposal of refined and updated heuristics –Simple and fast method to decompose traffic –No payload required –Effectively used even on short traces (10 min) 0.2% of the data left unclassified

Licentiate Seminar Wolfgang John Study IV: Classification Results (1) Tuesday,

Licentiate Seminar Wolfgang John Study IV: Classification Results (2) Application breakdown April till Nov. 2006

Licentiate Seminar Wolfgang John Study IV: Classification Results (3) Connection establishment for traffic classes

Licentiate Seminar Wolfgang John Study IV: Classification Results (4) Behavior of P2P traffic –Unsuccessful TCP connection attempts increasing –Serving peers terminate with FIN and RST Decreased from 20% to 8% –UDP overlay traffic doubled TCP options deployment differs –P2P behaves as expected –Web traffic shows artifacts of client-server patter e.g. popular web-servers neglecting SACK option

Licentiate Seminar Wolfgang John Summary 1.Guidelines for Internet measurement Experiences of the MonNet project 2.Current traffic characteristics Packet and flow level 3.Traffic decomposition Traffic classification method 4.Inconsistent behavior Packet header anomalies Malicious traffic flows

Licentiate Seminar Wolfgang John General remarks Internet today is essential, but still not understood entirely Large-scale traffic measurements uncommon –A lot of analysis is done on outdated datasets Each study generated as much questions as answers Reconsider measurement process (duration, payload…) A lot of open questions … …get more answers in two years…