Good Things Come In Small Packages Creative perfSONAR Antoine Delvaux PSNC — GÉANT perfSONAR Service Manager Linz — Austria 31 May 2017 You don’t always need big devices or powerful machines to improve things. We wanted to have more user adoption for perfSONAR, all we knew was: we had to be creative.

More perfSONAR? Get creative! What is this about? Small Nodes! Do you operate Small Nodes? As easy as can be! The Mesh, The Dashboard and The Colors You want examples? We’ve got examples! Useful? Really? So, tell me, what’s next?

TNC16: Small Nodes in GÉANT Launch Objectives: Increase perfSONAR adoption amongst GÉANT partners and users Early 2016: 50 european pS nodes in 1300 nodes worldwide (4 %) Show case the current perfSONAR Demonstrate perfSONAR usability on low cost devices Give first hand experience to new users, update the experience of old time users Increase the global perfSONAR footprint BoF session TNC16: Lots of interest 20 small nodes to 20 organisations 5 Universities 12 NREN (8 europeans + 3 africans + 1 asian) GÉANT + NORDUnet 1 outside the R&E community (RIPE NCC) pS adoption one year ago was still low More pS deployments benefits all pS users Packed room, and it was during lunch time!

A Small Node? Hardware Software The Small Nodes Low cost: around 200 € Capable of 1 gbps throughput Disk for local storage Hardware BRIX BACE-3150 devices Intel Celeron 4 cores Single 1 gbps NIC Software CentOS 6 perfSONAR toolkit 3.5.1 pre-configured DHCP setup Auto-updates (CentOS and perfSONAR) 200 € is low cost compared to thousands of € a fully fledged server can cost. To have 1 gbps need to have 1 gbps interface but also sufficient CPU power and RAM. Intel Celeron makes it easy to install regular CentOS.

The Small Nodes — Inside

Tell the group your node is online The Small Nodes — Setup 3 installation steps Plug the network Plug the power Turn it on 3 configuration steps Assign IP addresses (v4 and v6, preferably) Fill administrative details (Name, Country, Organisation, Geo info) Select closest NTP servers (automatically or manually) Tell the group your node is online Check forward and reverse DNS are functional and correct Quick Start guide distributed with devices — just in case Automatic Lookup Service registration 24 h later: first measurements Make it as easy as possible

Central Management of the mesh Managing the nodes Users and GÉANT Communication through mailing list A GÉANT SA2 activity — perfSONAR Service Monitoring SNMP polling, MRTG based Central Management of the mesh Mesh definition Central MA access Auto updates Managed by YUM CentOS + perfSONAR Can be skipped if too long. No alerting, don’t want to bother users/admins at this point. Should be reliable enough. Central MA access through IP based authentification.

New features and changes: perfSONAR 4.0 upgrade Released on April 17th New features and changes: New graphs pScheduler to replace BWCTL and regular testing daemon MaDDash 2.0 with email alerting CentOS7, full Debian support Recording of webinar: https://youtu.be/QS1CcbqOoJQ Small nodes in automatic upgrade mode Yum config Unattended upgrade All went well Dashboard New matrix: routes/paths

Controlled from central server Measurement Mesh Controlled from central server Single configuration file Measurement mesh agent on small nodes, automatically picks up mesh config changes IPv4 mesh 20 small nodes 5 GÉANT MP IPv6 mesh 16 small nodes Choice of GÉANT MP? Network / Geographical dispersion Not too loaded (chosen with GÉANT Ops advice) Reliable measurement points perfSONAR is most useful when regular measurements are scheduled Easier way to do that: setup a measurement mesh Centrally managed, so the small nodes users don’t need to be perfSONAR experts to start using pS GÉANT MP? Reliable MP both in terms of uptime/availability and of network and server performance.

Measurement Mesh — Visualisation Central dashboard: MaDDash Colored matrix showing latest measurement results and thresholds Detailed and historical graphs Configured measurements: Throughput up to 1 gbps Loss Latency: OWD Routes / Paths changes MaDDash is part of the perfSONAR distribution It provides a global overview of the network performance and helps quickly spot issues

Measurement Mesh — Example 1 First steps with mesh: Make sure all nodes are taking measurements Make sure all measurements are stored in the central MA Look for Orange squares: means no data Node unreachable No measurements happening Reason: Firewall issue Outgoing measurements ok Incoming measurements impossible First steps as a mesh administrator (us, GÉANT SA2) It is also possible to change dashboard colors. There is no example A, screen shot taken from the report.

Measurement Mesh — Example 2 Dashboard color meaning: Green: good performance Yellow: degraded performance Red: poor performance Orange: no data (other color scheme possible) Red square: poor performance Thresholds are indicated top of matrix Throughput < 500 Mbps Thresholds are configurable Reason: Saturated / Low bandwidth uplink

What can the users do with a Small Node? Owners can use the perfSONAR web interface Look at mesh measurements results Configure new measurements Owners can use the CLI On demand measurements Look at the way perfSONAR works Small Nodes are public Mesh measurements are public (dashboard) Small Nodes are listed on pS Lookup Service Can be target of measurements Getting more comfortable with perfSONAR and performance measurements in general Is it useful?

Loss measurements: GÉANT Budapest MP Yes, it is useful! One example of usefulness, happened just after a few weeks of operation. Dashboard showing losses to one GÉANT MP Happened just after a few weeks of operation

Details: Budapest MP to JANET Packet loss ~0.3% Loss rate is relatively small 0.3 % or 3 packets per 1000

Details: Budapest MP to PIONIER Packet loss ~0.3% Similar loss rate

Measurement mesh operations: some guidelines perfSONAR is a multi-domain performance monitoring tool Need to involve all parties MP operators, network operators GÉANT Budapest MP investigation example Contact with GÉANT Ops team Heavy traffic on the switch during the night (backup) Packet drops resulted Remedy: Connect perfSONAR MP directly to router Need to be in close relation with other partners in the mesh (here GÉANT Ops team) Some MP get replaced Some MP load increase We changed the measurement matrix from 4 MP to 5 MP If you use a foreign MP regularly, get in touch with the owners! Always remember that perfSONAR is made for multi-domain and the pSmall mesh is a multi-domain setup.

User Survey in November 2016 Outcomes: User Acceptance User Survey in November 2016 13 answers out of 20 project participants Outcomes: Overall average rating of 4.77 out of 5 100% of respondents recommends project continuation 100% of respondents recommends usage of perfSONAR on low cost devices 92% would like to have more GÉANT managed small nodes in their network 75% interested in installing and running perfSONAR on small nodes themselves 58% interested in installing and running fully-fledged perfSONAR server themselves 58% are interested in having their own measurement mesh in their network Survey was 6 months after the start of the project.

Increased perfSONAR coverage in Europe Tangible results Increased perfSONAR coverage in Europe 50 out of about 1300 early 2016 : 4% (estimation, from map) 275 out of 2093 August 2016: 13% (number of interfaces registered to LS) 557 out of 2231 May 2017 : 25% (number of interfaces registered to LS) About 10 times more pS deployments in 18 months! More user interest

Continuing activity and nodes management Extending the mesh What’s next? Continuing activity and nodes management Extending the mesh Objective: 1 Small Node per NREN Open to new self funded small nodes Ready to take part in the mesh? Subscribe to list at https://lists.geant.org/sympa/subscribe/perfsonar-smallnodes Announce yourself! Get in touch with us! Considering possibility to buy and distribute more Small Devices

antoine.delvaux@man.poznan.pl