1. University Computing Services EDUCAUSE Mid-Atlantic Regional Conference 16 January 2003 An Infrastructure and Accounting Response to Peer to Peer Traffic Volume Dr. Michael R Mundrane Director of Telecommunications Rutgers University Computing Services

2. University Computing Services Copyright Copyright Michael R Mundrane 2003. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

3. University Computing Services Agenda Statement of Problem Objectives Approach Results Conclusions

4. University Computing Services Statement of Problem Is he kidding? P2P is the problem!

5. University Computing Services Network Evolution Sporadic Unequally funded Unstructured Immediacy Complex Point services Faculty centric

6. University Computing Services Application Models Limited customer interface Minimal administration Centralized management Centralized storage hub and spoke infrastructure Minimal bandwidth TerminalHost

7. University Computing Services Application Models Rich customer interface Medium administration Centralized management Hybrid storage (server and client) Tiered network infrastructure Bandwidth server/s dependant ClientServer

8. University Computing Services Application Models Rich user interface High touch administration Distributed management (costly) Distributed storage (difficult to manage) Complex mesh infrastructure High bandwidth Peer

9. University Computing Services Cooperative?!? A. Badges? We don’t see no stinking badges! Q. Excuse me, would you please forward the business activity associated with your traffic so that we can adjust our records?

10. University Computing Services Objectives More than near term survival!

11. University Computing Services Essential Characteristics Preserve behavior Ensure access Moderate impact Protect privacy Avoid value judgments Apply to new applications

12. University Computing Services Assumptions Large number of hosts Small number of problems Service consumers Many random light providers Few heavy providers Responsive community

13. University Computing Services Just Use Traffic Shaping

14. University Computing Services Just Use Traffic Shaping Cisco routers CAR – traffic class MicroCAR – identified flow

15. University Computing Services Just Use QoS

16. University Computing Services Just Use QoS Classification Differentiation Admission control Provisioning Bandwidth Latency Jitter

17. University Computing Services QoS Differentiation P2P Other 10Mbit 90Mbit

18. University Computing Services QoS Differentiation 10Mbit Differentiation w/o admission control only defers the problem!

19. University Computing Services Rutgers Network 40,000+ hosts 1200+ networks 200+ routers 17 zones 7 campuses 3 regions 1 autonomous system

20. University Computing Services Approach No single solution!

21. University Computing Services Best Network Practices Modular Layered Aggregated Scalable Uniform Deterministic Comprehensible

22. University Computing Services Device Intra-building Backbone Building Intra-building Backbone RUNet ~ 1200

23. University Computing Services Building Inter-building Backbone Zone Inter-building Backbone RUNet 17

24. University Computing Services Zone Intra-campus Backbone Campus Intra-campus Backbone RUNet 7

25. University Computing Services Campus Inter-campus backbone Region Inter-campus Backbone RUNet 3

26. University Computing Services MAN Inter-region Backbone Autonomous System Inter-region Backbone RUNet 1

27. University Computing Services Characteristics Geographic independence Shallow topology Similar (not optimal) paths Low latency Uniform characteristics 1 autonomous system

28. University Computing Services Collect Data Netflow Source/Destination address Source/Destination ports Protocol Packets/Octets/Flows Start/End time

29. University Computing Services Raw Data 10 minute granularity Each source Each destination 1,000,000 addresses 10,000,000 records 1 Gigabytes, 1 day

30. University Computing Services Rollup Data Rutgers sources/sinks Data >= 1024, 10 minutes Data >= 6*1024, 1 hour Data >= 24*6*1024, 1 day 20,000 unique hosts 20,000 records 1 Megabyte

31. University Computing Services Filtered Data Rutgers sources/sinks Data >= 512 Megabytes, 1 Day 125 unique hosts 125 records 50 Kilobytes

32. University Computing Services Reduction

33. Distribution Reread entire data set Limit to filtered only Rollup based on external address Preserve individual distributions Useful to reduce contact

34. University Computing Services Questionable Distribution

35. University Computing Services Good Distribution

36. University Computing Services Storage Process Model Rollup Internet Netflow Filter Distribution Analyze

37. University Computing Services Residence Assumptions RFC1918 address space Large number of hosts Small number of problems Service consumers No service providers Unresponsive community

38. University Computing Services Set Limits 2048 MB download 512 MB upload 7 day granularity Sliding window Enforcement

39. University Computing Services Reference 4 movies 400 songs 45,000 web pages 2048 Megabytes

40. University Computing Services Oracle Process Model Table Rollup Table Enforce Table Gather Internet Netflow WWW Custom ACL

41. University Computing Services Traffic Shaping 1 Day on 7 Days off Multiplexed 1:8 ratio Automatic Aggregated Not legalistic Load Impact

42. University Computing Services Differentiated Service Residence facilities Other locations Two traffic classes 1:2 host distribution 1:1 bandwidth allocation CAR enforced

43. University Computing Services Results Some pains, some gains!

44. University Computing Services Extra Efforts Registration Port Address Translation Split horizon DNS Help desk/Appeals Address hopping Proxy services Oracle

45. University Computing Services 90% Data Sinks

46. University Computing Services 99.99% Data Sinks

47. University Computing Services 90% Data Sources

48. University Computing Services 99.99% Data Sources

49. University Computing Services Internet Traffic

50. University Computing Services Conclusions Modest applications with broad demographics have profound impact. Students have free time. Network best practices never more important. Cooperative generic methods can be effective (w/ encouragement). No magic bullet.

51. University Computing Services Questions? mundrane@td.rutgers.edu