1. 120 slides Mapping the Internet and Intranets Steve Branigan, Hal Burch, Bill Cheswick ches@lumeta.com

2. Mapping the Internet and intranets slide 2 of 120 Motivations Intranets are out of control – Always have been Highlands “day after” scenario Panix DOS attacks – a way to trace anonymous packets back! Internet tomography Curiosity about size and growth of the Internet Same tools are useful for understanding any large network, including intranets

3. Mapping the Internet and intranets slide 3 of 120 The Original Project Long term reliable collection of Internet and Lucent connectivity information – without annoying too many people Attempt some simple visualizations of the data – movie of Internet growth! Develop tools to probe intranets Extended database for researchers

4. Mapping the Internet and intranets slide 4 of 120 Uses for the Internet data topography studies long-term routing studies publicly available database – (“open source”) for spooks interesting database for graph theorists combine with other mappers to make an actual map of the Internet

5. Mapping the Internet and intranets slide 5 of 120 History of the Project Started in August 1998 at Bell Labs April-June 1999: Yugoslavia mapping July 2000: first customer intranet scanned Sept. 2000: spun off Lumeta from Lucent/Bell Labs June 2002: “B” round funding completed

6. Mapping the Internet and intranets slide 6 of 120 Related Work See Martin Dodge’s cyber geography page MIDS - John Quarterman CAIDA - kc claffy Mercator Enter “internet map” in your search engine

7. Mapping the Internet and intranets slide 7 of 120

8. Mapping the Internet and intranets slide 8 of 120

9. Mapping the Internet and intranets slide 9 of 120 Methods - data collection Single reliable host connected at the company perimeter Daily full scan of Lucent Daily partial scan of Internet, monthly full scan One line of text per network scanned – Unix tools

10. Mapping the Internet and intranets slide 10 of 120 Methods - network scanning Obtain master network list – network lists from Merit, RIPE, APNIC, etc. – BGP data or routing data from customers – hand-assembled list of Yugoslavia/Bosnia Run a traceroute-style scan towards each network Stop on error, completion, no data – Keep the natives happy

11. Mapping the Internet and intranets slide 11 of 120 Daily database 100-200MB of text compresses to 5-10MB daily Internet results available from mapping web page – have not checked to see who gets it! Saved to different partition, and offloaded to other secure computer

12. Mapping the Internet and intranets slide 12 of 120 Traceroute Probes toward each target network with increasing TTL Probes are ICMP, UDP, TCP to port 80, 25, 139, etc. Some people block UDP, others ICMP

13. Mapping the Internet and intranets slide 13 of 120 Traceroute Application level TCP/UDP IP Hardware Client IP Hardware Router IP Hardware Router IP Hardware Router IP Hardware Router IP Hardware Router Application level TCP/UDP IP Hardware Server Hop 1Hop 2 Hop 3 Hop 4

14. Mapping the Internet and intranets slide 14 of 120 Send a packet with a TTL of 1… Application level TCP/UDP IP Hardware Client IP Hardware Router IP Hardware Router IP Hardware Router IP Hardware Router IP Hardware Router Application level TCP/UDP IP Hardware Server Hop 1Hop 2 Hop 3 Hop 4

15. Mapping the Internet and intranets slide 15 of 120 …and we get the death notice from the first hop Application level TCP/UDP IP Hardware Client IP Hardware Router IP Hardware Router IP Hardware Router IP Hardware Router IP Hardware Router Application level TCP/UDP IP Hardware Server Hop 1Hop 2 Hop 3 Hop 4

16. Mapping the Internet and intranets slide 16 of 120 Send a packet with a TTL of 2… Application level TCP/UDP IP Hardware Client IP Hardware Router IP Hardware Router IP Hardware Router IP Hardware Router IP Hardware Router Application level TCP/UDP IP Hardware Server Hop 1Hop 2 Hop 3 Hop 4

17. Mapping the Internet and intranets slide 17 of 120 … and so on … Application level TCP/UDP IP Hardware Client IP Hardware Router IP Hardware Router IP Hardware Router IP Hardware Router IP Hardware Router Application level TCP/UDP IP Hardware Server Hop 1Hop 2 Hop 3 Hop 4

18. Mapping the Internet and intranets slide 18 of 120 Advantages We don’t need access (I.e. SNMP) to the routers It’s very fast Standard Internet tool: it doesn’t break things Insignificant load on the routers Not likely to show up on IDS reports We can probe with many packet types

19. Mapping the Internet and intranets slide 19 of 120 Limitations Outgoing paths only View is from scanning host only Takes a while to collect alternating paths Gentle mapping means missed endpoints Imputes non-existent links

20. Mapping the Internet and intranets slide 20 of 120 The data can go either way A EF D BC

21. Mapping the Internet and intranets slide 21 of 120 The data can go either way A EF D BC

22. Mapping the Internet and intranets slide 22 of 120 But our test packets only go part of the way A EF D BC

23. Mapping the Internet and intranets slide 23 of 120 We record the hop… A EF D BC

24. Mapping the Internet and intranets slide 24 of 120 The next probe happens to go the other way A EF D BC

25. Mapping the Internet and intranets slide 25 of 120 …and we record the other hop… A EF D BC

26. Mapping the Internet and intranets slide 26 of 120 We’ve imputed a link that doesn’t exist A EF D BC

27. Mapping the Internet and intranets slide 27 of 120 Remediations Alternate routes not a factor on intranets Scan from several sources “stitching” needed – Traceroute in different directions gives different interface IP addresses – Techniques needed to link multiple IP addresses to a single host machine

28. Mapping the Internet and intranets slide 28 of 120 Network scanning Custom program Concurrently scans towards 500 nets at once Throttled to 400 packets/sec – 100 p/s over dialup modems! Slow daily scan for host on destination network

29. Mapping the Internet and intranets slide 29 of 120 Data collection complaints Australian parliament was the first to complain List of whiners (25 nets) Military noticed immediately – Steve Northcutt – arrangements/warnings to DISA and CERT

30. Mapping the Internet and intranets slide 30 of 120 Visualization goals make a map – show interesting features – debug our database and collection methods – hard to fold up geography doesn’t matter use colors to show further meaning

31. Mapping the Internet and intranets slide 31 of 120

32. Mapping the Internet and intranets slide 32 of 120 “Peacock smashed on a windshield” - Dave Presotto Interesting art tantalizing edges interior shows ISPs (colored by IP address!) can’t trace routes can’t even find the probe host

33. Mapping the Internet and intranets slide 33 of 120 Colored by AS number

34. Mapping the Internet and intranets slide 34 of 120

35. Mapping the Internet and intranets slide 35 of 120

36. Mapping the Internet and intranets slide 36 of 120

37. Mapping the Internet and intranets slide 37 of 120

38. Mapping the Internet and intranets slide 38 of 120

39. Mapping the Internet and intranets slide 39 of 120

40. Mapping the Internet and intranets slide 40 of 120

41. Mapping the Internet and intranets slide 41 of 120

42. Mapping the Internet and intranets slide 42 of 120

43. Mapping the Internet and intranets slide 43 of 120

44. Mapping the Internet and intranets slide 44 of 120

45. Mapping the Internet and intranets slide 45 of 120

46. Mapping the Internet and intranets slide 46 of 120

47. Mapping the Internet and intranets slide 47 of 120

48. Mapping the Internet and intranets slide 48 of 120

49. Mapping the Internet and intranets slide 49 of 120

50. Mapping the Internet and intranets slide 50 of 120

51. Mapping the Internet and intranets slide 51 of 120 Map Coloring distance from test host IP address – shows communities Geographical (by TLD) ISPs future – timing, firewalls, LSRR blocks

52. Mapping the Internet and intranets slide 52 of 120 Colored by IP address!

53. Mapping the Internet and intranets slide 53 of 120 Colored by geography

54. Mapping the Internet and intranets slide 54 of 120 Colored by ISP

55. Mapping the Internet and intranets slide 55 of 120 Colored by distance from scanning host

56. Mapping the Internet and intranets slide 56 of 120 US military reached by ICMP ping

57. Mapping the Internet and intranets slide 57 of 120 US military networks reached by UDP

58. Mapping the Internet and intranets slide 58 of 120

59. Mapping the Internet and intranets slide 59 of 120

60. 120 slides Yugoslavia An unclassified peek at a new battlefield

61. Mapping the Internet and intranets slide 61 of 120

62. 120 slides Un film par Steve “Hollywood” Branigan...

63. Mapping the Internet and intranets slide 63 of 120

64. 120 slides fin

65. 120 slides NYC after 9/11

66. Mapping the Internet and intranets slide 66 of 120 CIDR and IP Counts

67. Mapping the Internet and intranets slide 67 of 120 Routers in New York City

68. Mapping the Internet and intranets slide 68 of 120 Internet before 9/11/2001

69. Mapping the Internet and intranets slide 69 of 120 Internet after 9/11/2001

70. 120 slides Let’s look at some intranets

71. Mapping the Internet and intranets slide 71 of 120

72. Mapping the Internet and intranets slide 72 of 120

73. Mapping the Internet and intranets slide 73 of 120

74. Mapping the Internet and intranets slide 74 of 120

75. Mapping the Internet and intranets slide 75 of 120

76. Mapping the Internet and intranets slide 76 of 120

77. 120 slides Anything large enough to be called an “intranet” is out of control

78. Mapping the Internet and intranets slide 78 of 120 This is not the fault of network administrators! Robust internet design frustrates central control – Ad hoc growth – Mergers and acquisitions frustrate long-term network planning and policies CIOs and auditors already know this

79. Mapping the Internet and intranets slide 79 of 120

80. Mapping the Internet and intranets slide 80 of 120

81. Mapping the Internet and intranets slide 81 of 120

82. Mapping the Internet and intranets slide 82 of 120

83. Mapping the Internet and intranets slide 83 of 120 We call these “routing leaks” Easily-found holes in the intranet perimeter Show up nicely on the maps Leaking hosts or routers announce routes to other networks or the Internet Sometimes left over from an old corporate split Non-functional VPNs can show up

84. Mapping the Internet and intranets slide 84 of 120 This was Supposed To be a VPN

85. Mapping the Internet and intranets slide 85 of 120 The maps are useful, but not the main data We collect tens of megabytes of network data There were unexpected subtleties to this process How do you display all this information, given that different clients want different data?

86. Mapping the Internet and intranets slide 86 of 120 The second technology: host leak detection Developed to find hosts that have access to both intranet and Internet Or across any privilege boundary Leaking hosts do not route between the networks May be a dual-homed host Not always a bad thing Technology didn’t exist to find these

87. Mapping the Internet and intranets slide 87 of 120 Possible host leaks Miss-configured telecommuters connecting remotely VPNs that are broken DMZ hosts with too much access Business partner networks Internet connections by rogue managers Modem links to ISPs

88. Mapping the Internet and intranets slide 88 of 120 Leak results Found home web businesses At least two clients have tapped leaks – One made front page news

89. Mapping the Internet and intranets slide 89 of 120 Leak Detection Prerequisites List of potential leakers: obtained by census Access to intranet Simultaneous availability of a “mitt”

90. Mapping the Internet and intranets slide 90 of 120 Leak Detection Layout Internet intranet Mapping host A Test host B mitt D C Mapping host with address A is connected to the intranet Mitt with address D has Internet access Mapping host and mitt are currently the same host, with two interfaces

91. Mapping the Internet and intranets slide 91 of 120 Leak Detection Internet intranet Mapping host A Test host B mitt D C Test host has known address B on the intranet It was found via census We are testing for unauthorized access to the Internet, possibly through a different address, C

92. Mapping the Internet and intranets slide 92 of 120 Leak Detection Internet intranet Mapping host A Test host B mitt D C A sends packet to B, with spoofed return address of D If B can, it will reply to D with a response, possibly through a different interface

93. Mapping the Internet and intranets slide 93 of 120 Leak Detection Internet intranet Mapping host A Test host B mitt D C Packet must be crafted so the response won’t be permitted through the firewall A variety of packet types and responses are used Either inside or outside address may be discovered Packet is labeled so we know where it came from

94. Mapping the Internet and intranets slide 94 of 120 Leak Detection Internet intranet Mapping host A Test host B mitt D C This describes outbound leaks Inbound leaks are usually much more serious

95. Mapping the Internet and intranets slide 95 of 120 Possible problems NAT egress filtering transit of sensitive data over the public networks

96. Mapping the Internet and intranets slide 96 of 120 Our new tools give new views of intranets The pictures are mostly for management Maps can show progress – red is bad, blue is good – we can color the maps in many ways The real value in the reports is the list of anomalies – network leaks, routing loops, open routers, etc.

97. Mapping the Internet and intranets slide 97 of 120 How we scan Via dialup, using RAS servers Secure tunnel, if you prefer – IP/SEC – PPTP – others?

98. Mapping the Internet and intranets slide 98 of 120 What we do Probe the network for things not in the official list Run a host enumeration Run leak tests on each host found

99. Mapping the Internet and intranets slide 99 of 120 Technology used Traceroute SNMP queries – Router type – Routing tables Pings Special leak detection probes – ICMP – UDP – Other possible if requested

100. Mapping the Internet and intranets slide 100 of 120 Report HTML-based Delivered on CDROM or DVD Maps Executive summary shows highpoints Interactive map viewer tool for Windows

101. Mapping the Internet and intranets slide 101 of 120 Competitors? Not yet, not quite – Many use the same terms, but offer different services Some components are pretty easy and free – Host enumeration – But we do it better (!) A bit like HP OpenView – OpenView doesn’t scale – Much slower

102. Mapping the Internet and intranets slide 102 of 120 Value Discovers unknown parts of the network Data feeds into existing tools, enhancing their value You can’t secure what you don’t know about Due diligence for intranets – Insurance? M&A activity Personnel turnover leaves legacy connections Business partners

103. Mapping the Internet and intranets slide 103 of 120 Getting a report Web-based We can send you a CD-ROM You can access a web server – FreeBSD-based – One-time password authentication – Very paranoid server

104. 120 slides Sample report

105. Mapping the Internet and intranets slide 105 of 120

106. Mapping the Internet and intranets slide 106 of 120

107. Mapping the Internet and intranets slide 107 of 120

108. Mapping the Internet and intranets slide 108 of 120

109. 120 slides Internet report

110. Mapping the Internet and intranets slide 110 of 120

111. Mapping the Internet and intranets slide 111 of 120

112. Mapping the Internet and intranets slide 112 of 120

113. Mapping the Internet and intranets slide 113 of 120

114. Mapping the Internet and intranets slide 114 of 120

115. Mapping the Internet and intranets slide 115 of 120

116. Mapping the Internet and intranets slide 116 of 120

117. 120 slides Intranet “Best current practices” We are acquiring the data to produce a paper: statistics over a variety of large intranets

118. Mapping the Internet and intranets slide 118 of 120 Some intranet statistics from Lumeta clients

119. Mapping the Internet and intranets slide 119 of 120

120. 120 slides Mapping the Internet and Intranets Steve Branigan, Hal Burch, Bill Cheswick ches@lumeta.com