1. Basic Configuration & Deployment
South Carolina State University SPAN Workshop
Matt Zekauskas,
This document is a result of work by the perfSONAR Project ( and is licensed under CC BY-SA 4.0 (

2. Host Considerations http://docs.perfsonar.net/install_hardware.html
Dedicated perfSONAR hardware is best
Server class is a good choice
Desktop/Laptop/Mini (Mac, Shuttle) can be problematic, but work in a diagnostic capacity
Other applications running may perturb results (and measurement could hurt essential services)
Running Latency and Throughput on the Same Server
If you can, devote 2 interfaces – the toolkit will support this.
If you can’t, note that Throughput tests can cause increased latency and loss (latency tests on a throughput host are still useful however)
© 2017, perfsonar.net
September 22, 2018September 22, 2018

3. Host Considerations http://docs.perfsonar.net/install_hardware.html
1Gbps vs 10Gbps testers
There are a number of problem that only show up at speeds above 1Gbps – both are still super useful
Virtual Machines do not always work well as perfSONAR hosts (use specific)
Clock sync issues are a bit of a factor
throughput is reduced significantly for 10G hosts
VM technology and motherboard technology has come a long way, YMMV
NAGIOS/SNMP/1G Throughput are good choices for a VM, OWAMP/10G Throughput are not
© 2017, perfsonar.net
September 22, 2018September 22, 2018

4. Toolkit Installation A CentOS netinstall is the quickest way to start
Others explained here
© 2017, perfsonar.net
September 22, 2018September 22, 2018

5. Toolkit Configuration
The best source of information is here:
There are two use cases for configuration:
Installation is done via the use of an entire distribution via ISO, or use of RPM packages
Diagnostic: Configure host to allow tests from others, don’t test on your own (e.g. beacon)
Permanent: Participate in a mesh, or configure tests as an island
© 2017, perfsonar.net
September 22, 2018September 22, 2018

6. Toolkit Configuration - Web
© 2017, perfsonar.net
September 22, 2018September 22, 2018

7. Toolkit Configuration - Web
We can do most other things via the web interface once we have a user that can auth against it (do-it-yourselfers can still hand edit config – we just won’t deal with that here)
Some things to check – for all see items under “perfSONAR Toolkit” on :
Administrative info
NTP (time keeping)
Turning on/off services
Configuring some tests (directly)
© 2017, perfsonar.net
September 22, 2018September 22, 2018

8. NTP
Scheduling, and one-way latency (OWAMP) depend on good knowledge of local time
Note that it may take a day to fully stabilize the clock
Pick 4 – 5 Close servers for NTP
We have a fast way to do this, or you can manually select
Can also add your own servers if you don’t like ours
© 2017, perfsonar.net
September 22, 2018September 22, 2018

9. Address Families
Note that the tools are IPv4 and IPv6 capable. If a functional IPv6 address is available, that will be the default chosen. To force one or the other in the test interface, pay attention to the test members:
To truly force one or the other, enter the address for a host, instead of a hostname.
© 2017, perfsonar.net
September 22, 2018September 22, 2018

10. Transition: Config to Regular Testing
perfSONAR web interface is meant to be simple
Enabling this on campus is the first step to seeing a simulation of performance for a bulk data tool. Ideally you would place the perfSONAR server where the users are (e.g if they are traversing a firewall still, why don’t you learn their pain)?
Configuring regular tests is systematic – pick regional and far away destinations.
Dust off netflow, and see where the data is going – configure tests to those locations too.
© 2017, perfsonar.net
September 22, 2018September 22, 2018

11. Regular Use
The best way to get buy in (at all levels) is to use the machines:
Encourage 1st line network support/help desk techs to see what the tools do.
Useful for basic things:
Traceroute beacon
Visualization of existing regular tests
Encourage 2nd line support to experiment with the command line tools.
Develop reports of performance (e.g. query data from the archive and create an XLS spreadsheet/chart)
© 2017, perfsonar.net
September 22, 2018September 22, 2018

12. Importance of Regular Testing
We can’t wait for users to report problems and then fix them (soft failures can go unreported for years!)
Things just break sometimes
Failing optics
Somebody messed around in a patch panel and kinked a fiber
Hardware goes bad
Problems that get fixed have a way of coming back
System defaults come back after hardware/software upgrades
New employees may not know why the previous employee set things up a certain way and back out fixes
Important to continually collect, archive, and alert on active throughput test results
© 2017, perfsonar.net
September 22, 2018September 22, 2018

13. Regular Testing There are a few schools of thought regarding this.
Beacon: Let others test to you (e.g. no regular configuration is needed)
Island: Pick some hosts to test to – you store the data locally. No coordination with others is needed. Can be done via web interface
Mesh: full coordination between you and others (e.g. consume a testing configuration that includes tests to everyone, and incorporate into a visualization)
© 2017, perfsonar.net
September 22, 2018September 22, 2018

14. Regular Testing - Beacon
The beacon setup is typically employed by a network provider (regional, backbone, exchange point)
A service to the users (allows people to test into the network)
Can be configured with Layer 2 connectivity if needed
If no regular tests are scheduled, minimum requirements for local storage.
Makes the most sense to enable all services (bandwidth and latency)
© 2017, perfsonar.net
September 22, 2018September 22, 2018

15. Regular Testing – Island
The island setup allows a site to test against any number of the perfSONAR nodes around the world, and store the data locally.
No coordination required with other sites
Allows a view of near horizon testing (e.g. short latency – campus, regional) and far horizon (backbone network, remote collaborators).
OWAMP is particularly useful for determining packet loss in the previous cases.
Throughput will not be as valuable when the latency is small
© 2017, perfsonar.net
September 22, 2018September 22, 2018

16. Regular Testing - Mesh A full mesh requires more coordination:
A full mesh means all hosts involved are running the same test configuration
A partial mesh could mean only a small number of related hosts are running a testing configuration
In either case – bandwidth and latency will be valuable test cases
© 2017, perfsonar.net
September 22, 2018September 22, 2018

17. Develop a Test Plan What are you going to measure?
Achievable bandwidth
2-3 regional destinations
4-8 important collaborators
4-8 (more if you are willing, especially to start) times per day to each destination
20-30 second tests within a region, longer across oceans and continents
Loss/Availability/Latency
OWAMP: ~10-20 collaborators over diverse paths
Interface Utilization & Errors (via SNMP)
What are you going to do with the results?
NAGIOS Alerts
Reports to user community
Dashboard
© 2017, perfsonar.net
September 22, 2018September 22, 2018

18. perfSONAR Deployment Locations
Critical to deploy such that you can test with useful semantics
perfSONAR hosts allow parts of the path to be tested separately
Reduced visibility for devices between perfSONAR hosts
Must rely on counters or other means where perfSONAR can’t go
Effective test methodology derived from protocol behavior
TCP suffers much more from packet loss as latency increases
TCP is more likely to cause loss as latency increases
Testing should leverage this in two ways
Design tests so that they are likely to fail if there is a problem
Mimic the behavior of production traffic as much as possible
Note: don’t design your tests to succeed
The point is not to “be green” even if there are problems
The point is to find problems when they come up so that the problems are fixed quickly
© 2017, perfsonar.net
September 22, 2018September 22, 2018

19. Sample Site Deployment
© 2017, perfsonar.net
September 22, 2018September 22, 2018

20. perfSONAR Dashboard - MaDDash
© 2017, perfsonar.net
September 22, 2018September 22, 2018

21. MaDDash and The Mesh Config
Measurement results are more useful when they can be “seen”, because this implies they will be acted on.
MaDDash is a software package that can be used to visualize the results of many perfSONAR tests
The Mesh Config is a way to manage multiple nodes, by giving them a uniform testing schedule
E.g. this is in contrast to the other method of configuration shown – manually entering our tests.
Changes node from ‘testing as an island’ to being a part of a larger testing strategy
Start Here:
© 2017, perfsonar.net
September 22, 2018September 22, 2018

22. Basic Configuration & Deployment
South Carolina State University SPAN Workshop
Matt Zekauskas,
This document is a result of work by the perfSONAR Project ( and is licensed under CC BY-SA 4.0 (