1. TEIN3 NOC Updates (Feb 2011 – Jul 2011)
Zhonghui Li
TEIN3 NOC Service Manager

2. Outline TEIN3 General Operation Updates
Traffic Analysis between TEIN3 and GEANT
Total Traffic over TEIN3 Backbone
Traffic Volume over TEIN3 Link
Yearly Operation Report for 2010
Collaboration with CAREN

3. TEIN3 General Operation Updates (Feb 2011 – Jul 2011)
TEIN3 Topology
Major Operation Issues
Trouble Ticket Statistics
Link Availability
Link Utilization
Current Status of Advanced Service

4. TEIN3 Topology TEIN3 BJ PoP  CNGI-6IX BJ PoP 1G  10G 622M  10G
(rate-limit 2.5G)
622M  10G
(rate-limit 2.5G)
AS4758  AS55824

5. Major Operation Issues
Upgraded HK-JP link from STM4 to 10GE
Upgraded HK-NICT link from GE to 10GE
Upgraded SG-JP link from STM4 to 10GE (rate-limit 2.5Gbps)
Upgraded SG-NICT link from GE to 10GE (rate-limit 2.5Gbps)
Migrated BJ-COP STM16 circuit from TEIN3 to CNGI-6IX
Finished BGP amendment for NKN (India) AS number revision (AS4758  AS55824)

6. Major Operation Issues
Finished the migration of TEIN3 BJ PoP (different racks in same room)

7. Major Operation Issues
Brought up IPv6 BGP peering with CAREN
Submitted the new version of TEIN3 Routing Guideline (v1.3)

8. Trouble Ticket Statistics

9. Trouble Ticket Statistics
No routing problem and other problem during last 6 months

10. Link Availability (w/o maintenance)

11. Link Availability (with maintenance)
SG-TW(GE) ~ 34%: Link has been decommissioned since 1 Apr
HK-PH(FE) & PH-MNL(DS3) ~ 87%: ASTI HK PoP router problem which has been solved since 20 Feb
HK-KR(STM16) ~ 93%: EAC cable cut from 20 Mar to 3 Apr
The average availability of all the other TEIN3 links is higher than 99.7%

12. Link Utilization (average)
Inbound > 15%: BJ-COP(2.5G), AU-PER(622M)
Outbound > 20%: HK-ID(155M), HK-HK(70M), SG-MY(155M), AU-PER(622M)

13. Link Utilization (maximum)
Maximum > 97%: BJ-HK(1G), HK-ID(155M), HK-HK(70M), SG-SG(45M), SG-TH(155M), AU-PER(622M), PH-MNL(45M), MB-LK(45M), MB-NP(45M)

14. Current Status of Advanced Service
TEIN3 NREN/Partner
BGP IPv4 Unicast
BGP IPv6 Unicast
PIM IPv4 Neighbor
BGP IPv4 Multicast
MSDP IPv4 Neighbor
PIM IPv6 Neighbor
BGP IPv6 Multicast
MPLS
BGP IPv4 Labeled Unicast
AARNet (AU)
Yes No
ASTI (PH)
CNGI-6IX (CN)
HARNET (HK)
ITB (ID)
KOREN (KR)
LEARN (LK)
MYREN (MY)
NCTU (TW)
NKN (IN)
NREN (NP)
PERN2 (PK)
SINET (JP)
SingAREN (SG)
ThaiREN (TH)
VinaREN (VN)
GEANT
APAN-JP
CAREN
TEIN3 JP POP
TEIN3 BJ/HK/SG/MB POP
No major change during last 6 months.

15. Traffic Analysis between TEIN3 and GEANT
2 links between TEIN3 and GEANT
BJ-COP (STM16)
MB-MAD (STM16)
Statistics based on cflow
Sample rate: 100:1
Time scale
BJ-COP: to
MB-MAD: to

16. Packet Length (BJ-COP)

17. Packet Length (MB-MAD)
The total percentage of small packets (<64B) and large packets ( B) over BJ-COP and MB-MAD links is about 80-90%, which is the major part of traffic.

18. AS Top 10 (BJ-COP, inbound)
Top 3 of Source AS: JANET/UK (AS786), RENATER/FR (AS2200), GARR/IT (AS137)
Top 3 of Destination AS: CSTNET/CN (AS7497), CERNET/CN (AS4538), KNU/KR (AS10052)
By source AS
By destination AS

19. AS Top 10 (BJ-COP, outbound)
Top 3 of Source AS: CSTNET/CN (AS7497), CERNET/CN (AS4538), KNU/KR (AS10052)
Top 3 of Destination AS: JANET/UK (AS786), RENATER/FR (AS2200), GARR/IT (AS137)
By source AS
By destination AS

20. AS Top 10 (MB-MAD, inbound)
Top 3 of Source AS: JANET/UK (AS786), GARR/IT (AS137), RENATER/FR (AS2200)
Top 3 of Destination AS: KNU/KR (AS10052), ARRNet/AU (AS7575), PERN/PK (AS45773)
By source AS
By destination AS

21. AS Top 10 (MB-MAD, outbound)
Top 3 of Source AS: ARRNet/AU (AS7575), PERN/PK (AS45773), ERNET/IN (AS2697)
Top 3 of Destination AS: JANET/UK (AS786), DFN/DE (AS680), GARR/IT (AS137)
By source AS
By destination AS

22. Protocol Top 10 (BJ-COP)
TCP: 80.2%
UDP: 15.4%

23. Protocol Top 10 (MB-MAD)
TCP: 81.0%
UDP: 18.2%

24. Service Top 10 (BJ-COP) Traffic Distribution:
84.9% Unknown (Grid, P2P etc.)
14.2% WWW
0.2% DNS
0.2% FTP
0.2% Telnet
0.1% (SMTP)

25. Service Top 10 (MB-MAD) Traffic Distribution:
76.6% Unknown (Grid, P2P etc.)
21.7% WWW
0.5% DNS
0.5% (SMTP)
0.3% FTP
0.2% (POP3)

26. Address Block Top 10 (BJ-COP, inbound)
JANET, RENATER, KIT
CSTNET, KNU, CERNET
By source IP
By destination IP

27. Address Block Top 10 (BJ-COP, outbound)
CSTNET, CERNET, KNU
GARR, JANET, KIT
By source IP
By destination IP

28. Address Block Top 10 (MB-MAD, inbound)
RENATER, JANET, GARR
KNU, PERN, AARNet
By source IP
By destination IP

29. Address Block Top 10 (MB-MAD, outbound)
PERN, ERNET, University of Melbourne
GARR, SUNET, CERN
By source IP
By destination IP

30. Total Traffic over TEIN3 Backbone (inbound, TBytes)

31. Total Traffic over TEIN3 Backbone (outbound, TBytes)
The decrease of total traffic (both inbound and outbound) since Jun 2011 is due to the migration of BJ-COP from TEIN3 to CNGI-6IX, i.e. the re-direction of EU-CN traffic

32. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
BJ-2G_to_CN

33. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
HK-2.5G_to_KR

34. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
HK-155M_to_VN

35. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
HK-155M_to_ID

36. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
HK-70M_to_HK

37. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
HK-22.5M_to_PH

38. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
HK-1G_to_CAREN

39. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
SG-155M_to_MY

40. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
SG-155M_to_PK

41. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
SG-45M_to_SG

42. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
SG-1G_to_AU

43. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
SG-155M_to_TH

44. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
SG-155M_to_TW

45. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
MB-10G_to_IN

46. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
MB-2.5G_to_EU-MAD

47. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
MB-45M_to_LK

48. Traffic Volume over TEIN3 Link (inbound/outbound, TBytes)
MB-45M_to_NP

49. Yearly Operation Report for 2010
Trouble Ticket Statistics
Link Availability
Link Utilization
Comparison with 2009

50. Trouble Ticket Statistics
Tickets opened in 2010
Totally 271
Outage: 168
Maintenance: 93
Routing Problem: 1
Other Problem: 9

51. Trouble Ticket Statistics

52. Link Availability

53. Link Utilization (average)

54. Link Utilization (maximum)

55. Comparison with 2009 2009 2010 PoP 5 Backbone Link 4
(SG-MB 2.5G)
Europe-Asia Intercontinental Link
2 x 2.5G
(SG-LON  MB-MAD)
NREN 13 16
(India, Sri Lanka, Nepal)
Partner Network 2 3
(CAREN)
Total Traffic
(in/out, TB) / /
( increased by 76%/75%)
BGP Prefix
(IPv4/IPv6)
~12100/~390
~12500/~620
(increased by 3%/59%)
Advanced Service
IPv4/IPv6 inter-domain multicast
IPv4/IPv6 inter-domain multicast, Inter-domain MPLS/L2VPN

56. Collaboration with CAREN
Allocated rack space for CAREN M7i router at TEIN3 HK PoP
Cooperated with CAREN NOC on CAREN routing policy design
Provided on-site support for CAREN M7i installation in MEGA-I (DANTE contract)
Provided remote console access to CAREN M7i for CAREN NOC via TEIN3 HK PoP 2811 access router
Brought up HK-CAREN GE link
The first BGP peer to TEIN3 backbone with 4-byte AS number (AS197118)
Provided L2VPN local-switching for CAREN NREN’s FE access circuit (HK-Ashgabat) termination in MEGA-I via TEIN3 HK PoP EX3200 switch and M120 router (due to CAREN M7i’s lack of additional FE & GE interface)
Providing ongoing support for CAREN operations (DANTE contract)

57. Virtual link implementation for CAREN over TEIN3 backbone

58. Thanks!