1. Tools for TEIN2 operation Yoshitaka Hattori Jin Tanaka APAN-JP

2. Which tools? NREN NOC decides the tools by herself. –This presentation is only our proposal But we hope the other NREN NOCs can get the remote measurement data via web page – traffic graph, raw data… –At least operators and researchers –Hopefully, to public, if allowed Which tools are expected for the common data exchange? –Use existing well-known tools There are many open sources for installing and configuring them Well-known data formats are useful for both operators and researchers –Development of new tools expected for TEIN2 Provide the function that cannot be supported by well-known tools Might require more human resources and cost

3. Well-known Tools FunctionTools Required machine Install at NOC only Install at POP(s) Install at all nodes Remarks Tracking Request Tracker or other bug tracker system Machine A FreeBSD ☆☆☆☆ ★★★ -- To be installed at NOC only for managing trouble and other operation work Usage data MRTG or RRDtool + Frontend Machine A ☆☆ ☆☆☆☆ RRDtool + cricket is FreeBSD ★★★★ recommended for traffic Animated Traffic MAP Machine A ☆☆☆ FreeBSD ★★★ Perform ance data Throughp ut data Iperf & BWCTL Machine B- ☆☆☆☆☆☆☆☆ NTP source is required To be installed at least two locations Linux- ★★★★★★★★ RTT & Packet loss data Smoke Ping Machine A ☆☆ ☆☆☆ FreeBSD ★★★★ One way latency data OWAMP Machine C- ☆☆☆☆☆☆☆ NTP source is required To be installed at least two locations FreeBSD- ★★★★★★★★ Flow analysis flow-tools (NetFlow) Machine A ☆☆☆☆☆☆☆☆ Depending on router capability FreeBSD ★★★★★★★★ Equipment status Router data IU Router Proxy or Looking-glass Machine A ☆☆☆☆☆☆☆☆ FreeBSD ★★★ ☆ Useful for advanced operation ☆ :Lowest ☆☆☆☆☆ :Highest ★ Man-hours and cost ★ :Lowest ★★★★★ :Highest

4. Machines Machine A for measurements (traffic etc.) –It will be installed on NOC (and some POPs) –FreeBSD or Linux based, RRDtool+cricket, Animated Traffic Map, SmokePing, Flow-tools and IU Router Proxy Machine B for throughput performance –Linux with tuned kernel for TCP, iPerf and BWCTL –It will be installed on each POP (and some nodes, if we can allow its cost) Machine C for One-way latency performance –FreeBSD or Linux based, OWAMP –NTP MUST be configured –It will be installed on each POP (and some nodes, if we can allow its cost) In minimum case –At least one Machine A must be installed in NOC for measurements –Machine B and C are optional

5. Figure To EU TransPAC2 to US NOC Collecting Traffic and Flow Data Machine A Machine B&C HK SG JP Machine B&C Throughput Performance Measurement One way latency Measurement Machine B To EU Machine C POPs Nodes Machine A BJ Machine B&C APAN -JP TEIN2-North TEIN2-South TEIN2-JP

6. How to administrate tools? There are two possible schemes 1.All tools and servers are administrated only by NOC May need more human resources at NOC Difficult to manage remotely-placed servers 2.NOC administrate them in cooperation with organizations at each POP Tools and their data are administrated by NOC The organizations may provide remote-hand service Server hardware support and its OS administration may be provided by the organizations, if they allow its cost NOC may provide technical support to them

7. Implementation plan (proposal) Firstly, develop server and implement standard tools such as traffic measurement at NOC, and open the data –At least one Machine A at NOC Secondly, install advanced tools such as Router Proxy on NOC (and some POPs, if its cost is allowed) –Install them on Machine A –In minimum case, these tools may be installed only at NOC Finally, install the servers for advanced tools such as iPerf or OWAMP at each POP and some nodes, if its cost is allowed –Machine B and C

8. Links Bug Tracking systems –http://www.bestpractical.com/rt/ –http://www.mantisbt.org/ –http://www.daifukuya.com/kagemai/ (in Japanese only) MRTG, RRDTool and cricket: –http://people.ee.ethz.ch/~oetiker/webtools/mrtg/ –http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/ –http://cricket.sourceforge.net/ Animated Traffic Map: –http://loadrunner.uits.iu.edu/weathermaps/abilene/ iPerf & BWCTL: –http://dast.nlanr.net/Projects/Iperf –http://e2epi.internet2.edu/bwctl/ Smoke Ping: –http://people.ee.ethz.ch/~oetiker/webtools/smokeping/ OWAMP: –http://owamp.internet2.edu/ Flow-tools: –http://www.splintered.net/sw/flow-tools/ IU Router Proxy and Looking-glass variants: –http://tools.globalnoc.iu.edu/routerproxy.html –http://www.traceroute.org/#source%20code Very useful tools link on Grobal NOC –http://tools.abilene.iu.edu/

9. Thank you

10. Appendix. Server specification Machine A (for measurements) –OS: FreeBSD –Needs enough processing capability, disk and memory space Dual processors are recommended, if you can allow its costs RAID 1 or 5 system is recommended to rescue data from disk failure – Middle-range server such as SuperMicro 6023P-8R (2U, 6 disk slots, redundant P/S) About $2,400 for minimum configuration (Xeon 2.8GHz single, 1GB MEM, 72GB Disk), over $3,000 for dual processor It may be good idea to rent one from your vendor (DELL or HP...) – Low-end server Will enable you to reduce rack space and get more stability than standard PC –  Standard PC system Less than $700 (3GHz class processor, 1G MEM, 250GB Disk) Installing in standard office room is *NOT* good idea –High temperature without air conditioning may cause equipment failure (mostly disk failure) More rack space may be required More stability, performanceLower cost

11. Appendix. Server specification (cont.) Machine B (for throughput performance) –OS: Linux with tuned kernel (web100) –Needs enough processing capability, memory and I/O bandwidth Processing capability may be enough with single processor, I/O bandwidth is important –Intel or Broadcom Gigabit NIC connected via PCI-X or PCI-Express bus Onboard or additional card Legacy 32bits 33MHz PCI bus can’t provide enough bandwidth for 1Gbps –We can use cheap IDE disk, because disk speed is not significant – Middle-range server such as SuperMicro 6013P-8 (1U, 3 disk slots) Onboard dual Intel PCI-X based Gigabit NIC (1000Base-T) It costs about $2,000 for minimum configuration (Xeon 2.8GHz single, 1GB MEM, 72GB Disk) It may be good idea to rent one from your vendor (DELL or HP...) – Low-end server with PCI-X or PCI-Express based onboard Gigabit NIC Will enable you to reduce rack space and get more stability than standard PC –  PCI-Express based standard PC system Only less than $800, good for minimizing cost but certain rack space may be required Carefully select the main board, when using onboard Gigabit NIC –Some onboard Gigabit NIC may provide poor performance and stability –Intel or Broadcom connected via PCI-Express will be nice More stability, performanceLower cost

12. Appendix. Server specification (cont.) Machine C (for One way latency performane) –OS: FreeBSD or Linux –Needs NTP synchronization – Low-end 1U server Will enable you to reduce rack space and get more stability than standard PC Low-end processor can provide enough processing capability – Standard PC system Less than $600 (Mid-range processor, 512M MEM, 40GB Disk) Installing in standard office room is *NOT* good idea –High temperature without air conditioning may cause equipment failure (mostly disk failure) More rack space may be required More stability, performanceLower cost