1. Licentiate Seminar: On Measurement and Analysis of Internet Backbone Traffic Wolfgang John Department of Computer Science and Engineering Chalmers University of Technology Göteborg, Sweden

2. 2008-02-29Licentiate Seminar Wolfgang John Internet, 1983 Internet, 2005 Why measure Internet traffic? (1) The Internet is changing in size ARPANET, 1969

3. 2008-02-29Licentiate Seminar Wolfgang John The Internet is changing in application Why measure Internet traffic? (2)

4. 2008-02-29Licentiate Seminar Wolfgang John The Internet –is constantly developing –is used differently in different locations –is heterogeneous The Internet is not understood in its entirety! INTERconnected NETworks Why measure Internet traffic? (3) INTER NET

5. 2008-02-29Licentiate Seminar Wolfgang John Operational purpose –Troubleshooting, provisioning, planning …. Scientific purpose –Protocols, infrastructure and services –Performance properties –Internet simulation models –Security measures Why measure Internet traffic? (4)

6. 2008-02-29Licentiate Seminar Wolfgang John Thesis Objectives 1.Guidelines for Internet measurement 2.Current traffic characteristics 3.Traffic decomposition 4.Inconsistent behavior

7. 2008-02-29Licentiate Seminar Wolfgang John Outline Measurement approaches Internet measurement challenges The MonNet project Scientific contribution Results –Four studies included Conclusions Measurement Analysis

8. 2008-02-29Licentiate Seminar Wolfgang John Measurement approaches Network traffic measurement Active Passive Software Hardware Online Offline Flows Packets Complete Headers Different protocol levels Statistical summaries Transport layer

9. 2008-02-29Licentiate Seminar Wolfgang John Internet measurement challenges (1) Legal considerations Ethical and moral considerations Operational considerations Technical considerations

10. 2008-02-29Licentiate Seminar Wolfgang John Measurement challenges (3) Technical considerations Data amount –Exhausting I/O and storage access speeds Data reduction techniques –Filtering, sampling, packet truncation Timing –Clock synchronization

11. 2008-02-29Licentiate Seminar Wolfgang John The MonNet Project (1) Technical Solution 10 Gbps Göteborg splitter Borås 10 Gbps Processing Platform and Storage Measurement Node 2 Measurement Node 1

12. 2008-02-29Licentiate Seminar Wolfgang John The MonNet Project (2) Internet Regional ISPs Göteborg Stockholm Other smaller Univ. and Institutes Göteborgs Univ. Student- Net Chalmers Univ. Measurement location Borås April 2006 148 traces (20 minutes) 11 billion packets, 7.6 TB of data Sept. – Nov. 2006 554 traces (10 minutes) 28 billion packets, 19.5 TB of data

13. 2008-02-29Licentiate Seminar Wolfgang John Scientific Contribution Level of complexity Quantification of inconsistent behavior Traffic characterization Packet level Flow level Traffic classes Study I Study II Study IV Study III Upcoming

14. 2008-02-29Licentiate Seminar Wolfgang John Study I: Packet Level Analysis Updated packet-level characteristics of Internet traffic Inconsistencies in headers will appear –Network attacks and malicious traffic –Active OS fingerprinting –Buggy applications or protocol stacks

15. 2008-02-29Licentiate Seminar Wolfgang John High level analysis does not necessarily show differences → detailed analysis does! 2 main reasons for directional differences: –Malicious traffic the Internet is “unfriendly” –P2P Göteborg is a P2P source P2P is changing traffic characteristics e.g. packet sizes, TCP termination, TCP option usage Study II: Flow level analysis

16. 2008-02-29Licentiate Seminar Wolfgang John Study III: Classification Method (1) Classification of flow traffic without payload Heuristics to identify nature of endpoints Rules based on connection patterns and port numbers –5 rules for P2P traffic –10 rules to classify other types of traffic remove ‘false positives’ from P2P

17. 2008-02-29Licentiate Seminar Wolfgang John Study III: Classification Method (2) # connections in 10 6 Amount of data in TB Comparison of classification methods for P2P traffic

18. 2008-02-29Licentiate Seminar Wolfgang John Study III: Classification Method (3) Previous classification methods on packet header traces don’t work well on backbone data Proposal of refined and updated heuristics –Simple and fast method to decompose traffic –No payload required –Effectively used even on short traces (10 min) 0.2% of the data left unclassified

19. 2008-02-29Licentiate Seminar Wolfgang John Study IV: Classification Results (1) Tuesday, 18.04.2006

20. 2008-02-29Licentiate Seminar Wolfgang John Study IV: Classification Results (2) Application breakdown April till Nov. 2006

21. 2008-02-29Licentiate Seminar Wolfgang John Study IV: Classification Results (3) Connection establishment for traffic classes

22. 2008-02-29Licentiate Seminar Wolfgang John Study IV: Classification Results (4) Behavior of P2P traffic –Unsuccessful TCP connection attempts increasing –Serving peers terminate with FIN and RST Decreased from 20% to 8% –UDP overlay traffic doubled TCP options deployment differs –P2P behaves as expected –Web traffic shows artifacts of client-server patter e.g. popular web-servers neglecting SACK option

23. 2008-02-29Licentiate Seminar Wolfgang John Summary 1.Guidelines for Internet measurement Experiences of the MonNet project 2.Current traffic characteristics Packet and flow level 3.Traffic decomposition Traffic classification method 4.Inconsistent behavior Packet header anomalies Malicious traffic flows

24. 2008-02-29Licentiate Seminar Wolfgang John General remarks Internet today is essential, but still not understood entirely Large-scale traffic measurements uncommon –A lot of analysis is done on outdated datasets Each study generated as much questions as answers Reconsider measurement process (duration, payload…) A lot of open questions … …get more answers in two years…