SURFnet6 Network Monitoring and Reporting Hans Trompert, SURFnet

Information needs Connected organizations NOC / SURFnet / research Annual report Information detail

Monitoring versus Reporting -Monitoring -real-time -status -alarms -Reporting -afterwards -over a specific time period (day, week, month, year)

Information source and destination Avici SSR Nortel ERS8600 Nortel OM5200 Nortel OME6500 Nortel OME1060 SURFnet6 operations Real-time customer reporting Security

Equipment and interface Optical devicesCPLTL1 OM5200TL1 (+ SNMP) OME6500TL1 (+ SNMP) OME1060SNMP Data devicesERS8600SNMP Avici SSRSNMP + Netflow

Reporting: SNMP metrics SNMP metrics: -Interface in/out octet counters -Interface in/out packet counters (unicast/broadcast/multicast) -Interface input/output errors -Interface availability -Temperature -Memory -CPU -Device uptime -and more …

Reporting: TL1 metrics TL1 metrics: -Input/Output Frames -Errored frames -Discarded frames -Transmit and receive power levels -Errored Seconds - number of seconds that have had CRC errors -Severely Errored Seconds - after 10 seconds of ES we start counting SES -UnAvailable Seconds - Seconds where we had no sync -and more …

Monitoring: SNMP traps SNMP traps -Fan -Temperature -Voltage -Link Up/Down -Bay Controller -Module -PIM + MSDP -BGP -VRRP -ISIS -and more …

Monitoring: TL1 events TL1 Events -Equipment -Circuit pack missing/mismatch/failed -Fan failed/missing -Power failure A or B -High temperature -Shelf -Software upgrade failed/mismatch/…. -Database integrity fail/restore in progress/… -Amplifier -input/output loss of signal -automatic shutoff -and many, many more

SNMP based volume reporting Internet Connected organizations Border router Amsterdam1 (SARA) Border router Amsterdam2 (TeleCity II) Core router Amsterdam2 (TeleCity II) Core router Amsterdam1 (SARA) -Total external traffic -Per traffic class (AMS-IX, Global, privat peers) -Per provider/peer -Total SURFnet internal traffic -Per connected organization

SURFnet external traffic volume -SURFnet external traffic volume -Ams-IX -Private peers (via Ams-IX), including: -Chello, Planet, XS4all -Garnier Projects, Abovenet, UUnet, Cogent -NREN -Geant2 -SINET -Abilene -Global -Global Crossing -Cable & Wireless

SURFnet external traffic volume

SURFstat: Real-time connected organization traffic volume reporting -Software -Net-SNMP -Python -RRDtool -Features -Easy administration by labeling connections with keywords in interface description on router -Different graph resolutions: day, week, month, year, decade -1 minute measurement interval -Reports on -volume (bits in/out) -packets (unicast/multicast/broadcast)

SURFstat: UvA (many users)

SURFstat: CWI (few users)

Netflow – flow information -Netflow uses the common 5-tuple definition, where a flow is defined as a unidirectional sequence of packets all sharing all of the following 5 values: 1.Source IP address 2.Destination IP address 3.Source TCP port 4.Destination TCP port 5.IP protocol -Most common fields in Netflow record: -5-tuple information -Input and output SNMP interface index -Timestamps for the flow start and finish time -Number of bytes and packets observed in the flow

Netflow – versions v1 First try v5 Most used version v6 Encapsulation information v7 Switch information v8 Several aggregation forms v9 Template Based, allowing many combinations, supports IPv6 IPFIX aka v10; IETF Standardized NetFlow 9 with Enterprise fields and other community input

Netflow setup Internet Connected organizations Border router Amsterdam1 (SARA) Border router Amsterdam2 (TeleCity II) Core router Amsterdam2 (TeleCity II) Core router Amsterdam1 (SARA) FLOWmon perfSONAR test NFSEN PeakFlow Fan out

Netflow applications -connected organizations: -FLOWmon detailed traffic reporting -SURFflow (Arbor Peakflow / NFSEN) suspicious traffic pattern reporting -SURFnet-CERT: -NFSEN suspicious traffic pattern reporting historical flow data queries profiles for custom reports -Geant2 JRA1 perfSONAR probes -Flow Subscription Measurement Point -Flow Selection and Aggregation Measurement Archive

FLOWmon Detailed traffic reporting: -total traffic -prefix-based flow grouping -reports on: -IP version (v4/v6) -IP protocol (TCP, UDP, ICMP, GRE, …) -TCP port (HTTP, SMTP, NNTP, FTP, SSH, …) -UDP port (domain, RTSP, VPN, …) -top N connected organizations -destination AS traffic

UvA traffic by IP protocol

Connected organization to world traffic by TCP destination port

SURFflow Reports on suspicious traffic patterns like: -Unusual amount of flows  DOS attack -Flows from one host to many ports on other host  portscan -From 1 host to same port on many hosts  break- in attempt making use of known bug -From many hosts to specific (set of) port(s) to many other hosts  virus/worm -etc …

Active measurements: RTTPL Round Trip Time and Packet Loss monitoring -measurement probes throughout the network -central storage of results -active measurements by injecting ICMP echo request packets -measuring min/max/avg RTT and jitter -both IPv4 and IPv6 -both unicast and multicast (under development) -measuring packet loss -20 pings per minute -report matrices per minute/hour/day/month -results between two probes in graphs

RTTPL report matrices

RTTPL Nijmegen - Amsterdam

Active measurements: Connected organization availability -measuring availability by sending ICMP Echo Requests to connected organization router -measurement includes last mile to connected organization plus connected organization router port (unlike commercial providers) -Cisco routers with Service Assurance Agent software on both Amsterdam1 and Amsterdam2 -results stored in database and reported monthly -redundancy in measurements by ORing results from Amsterdam1 and Amsterdam2

Thank you