1. Monitoring Persistently Congested Internet Links
Leiwen (Karl) Deng
Aleksandar Kuzmanovic
Northwestern University

2. Objective
New probing methods that can improve measurement observability for core congestion
Pong – a tool specialized in measuring a subset of non-edge links exhibiting repetitive congestion
Can reveal systematic problems such as routing pathologies, poorly-engineered network policies, or non-cooperative inter-AS relationships
Lightweight: monitoring in addition to on-demand measuring
A building block of a large scale triggered monitoring system for Internet congestion

3. Repetitive Congestion
We focus on locating and monitoring non-edge links that exhibit repetitive congestion
Queuing delay as congestion indicator
Queue building-up repetitively happens on time scales of one or more minutes.

4. Methodology Highlights
Coordinated probing
Probe from both endpoints of a path
Combine end-to-end probes with (TTL limited) probes to intermediate routers
Infer underlying path topology conditions
Implicit inference
Based on measured queuing delays on different probing paths
Use statistics over longer time scales
Quantify measurement accuracy
Link measurability score
Probing from both endpoints of a path (to improve congestion observability).
Combine end-to-end probes with (TTL limited) probes to intermediate routers
Path topology conditions:
For example, when probing from destination endpoint to an intermediate node … returning path from the intermediate node … takes the same route as ...
Based on measured queuing delays on different probing paths
(take advantage of the repetitive congestion pattern)
For example, if two path observe the same … will match some condition; if there is a repetitively link locate at places … do not share, then … very inconsistent
Use statistics over longer time scales (to identify congestion location)
Quantify measurement accuracy
clock skew, route alterations, undesirable path conditions

5. A Simplified Case – Symmetric Path
Coordinated Probing
Probe f S s D d b
f (“forward”) probe ,
b (“backward”) probe ,
s (“source”) probe ,
d (“destination”) probe
Here we use a symmetric path scenario as an example.
In this scenario, for each intermediate node, we send four probe packets.
A Simplified Case – Symmetric Path

6. Half-path queuing delay
Coordinated Probing
Probe Δf f Δd S s D d Δs b Δb
Locating
Congested
Links
Tracing
Congestion
Status
Half-path queuing delay
Δfs
Δfd

7. Infer Underlying Path Topology Conditions
Probe Δf f Δd S s D d Δs b Δb
Condition: Δf +Δb ≈Δs +Δd
Path Pattern: 4-p probing scenario

8. Estimates of half-path queuing delay
Infer Underlying Path Topology Conditions
Probe
Probe S D f s b d
Paired d probe
Congestion
4-p probing
Pair up S D f s b d
Observed by b probe only
Paired d probe
Congestion
Pair up
Fsd probing
Probe S D f s b
No suitable d probes to pair up with this s probe
Congestion
Fsb probing
Probing technique
Estimates of half-path queuing delay
Condition
Δfs
4-p probing
Δf +Δb ≈Δs +Δd
In our implementation, one important thing is how can we decide which scenario we are in and which probing method we should use.
Actually, we achieve this by checking certain condition of the measured queuing delays.
Δfd
Δf ≈Δs +Δd
Fsd probing
Δs ≈Δf +Δb
Fsb probing
unconditional
2-p probing

9. Select Probing Techniques
Adjust probing technique online based on quality of measurability (QoM)
Probing technique
Condition
Definition of QoM
max(Δf +Δb, Δs +Δd)
QoM4p = 1 −
|(Δf +Δb) − (Δs +Δd)|
4-p probing
Fsd probing
Fsb probing
2-p probing
Promote
Δf +Δb ≈Δs +Δd
Δf ≈Δs +Δd
Δs ≈Δf +Δb
unconditional
Demote
max(Δf, Δs +Δd)
QoMfsd = 1 −
|(Δf − (Δs +Δd)|
QoMfsb = 1 −
|(Δs − (Δf +Δb)|
max(Δs, Δf +Δb)
To check whether a condition is matched and to how well it is matched, …
(Last resort)

10. Locating Congested Links
Perform coordinated probing for all intermediate nodes
Probe
Probe
Probe
Probe
Probe S D
Δfs
Δfs
Δfs
Δfs
Δfs
Δfd
Δfd
Δfd
Δfd
Δfd
For each path, we probe each node along the path sequentially.
This is called sequential probing mechanism.
Please note that if we are only interested in locating congestion points, we would prefer to sending all probes at the same time.
However, since we want to improve the overall probing rate by exploiting probes to different nodes, we disperse their probing time a little bit.
That’s why we use sequential probing.
Probe all nodes simultaneously

11. Locating Congested Links
Switch Point Approach
Congestion
Probe
Probe S D
Congested link is identified
Correlate probes to neighboring nodes
If the probing results of neighboring nodes show certain kind of pattern, we may conclude that the link between these two nodes is congested.
For example …
A switch point is a candidate congestion point.
Each time a switch point is detected, we update a metric called congestion count for this switch point.
Congestion count indicates the probability of congestion.
When congestion count exceeds certain threshold, we regard the point as a congestion point.

12. Tracing Congestion Statusf f f f f f f f f f f f f f f S D
Link C (Identified congested link)
Congestion Status
Link C
By reusing probes sent to different intermediate nodes, we can trace congestion status of detect congestion points.
Time
Use fast rate end-to-end probing

13. Emulab Experiment Example1 2 3 4 5 6 7 8
0.37s on/off
0.71s on/off
0.53s on/off
0.47s on/off
0.83s on/off 12 11 10 9
Topology: 12 nodes (PCs), 11 links
Link: 100Mbps, 2ms
Cross traffic: Each consists of 3 parallel TCP flows, 50% time on and 50% time off.
Build multiple bottlenecks: Cross traffics are added to corresponding links concurrently.

14. Emulab Experiment Example1 2 3 4 5 6 7 8
0.37s on/off
0.71s on/off
0.53s on/off 12 11 10 9
At the Beginning

15. After adding backward bottlenecks
Emulab Experiment Example 1 2 3 4 5 6 7 8
0.37s on/off
0.71s on/off
0.53s on/off
0.47s on/off
0.83s on/off 12 11 10 9
After adding backward bottlenecks

16. After adding two more forward bottlenecks
Emulab Experiment Example 1 2 3 4 5 6 7 8
0.37s on/off
0.71s on/off
0.53s on/off 12 11 10 9
0.29s on/off
0.63s on/off
After adding two more forward bottlenecks

17. Optimizing Pong in the Internet
Set queuing delay threshold
Based on distribution of queuing delay samples
Tune other parameters based on experiments on the PlanetLab
Minimize measurement errors
Detect and react to anomalies (clock skews, router alterations, ICMP queuing, etc)
Use instantaneous quality of measurement value as sample weight
Quantify measurement quality
Help select vantage points

18. Quantify Measurement Quality
Help select vantage points
Link measurability score
Probing technique and quality of measurability
Queuing delay threshold quality
Observability score
Congestion observed on a less frequently congested link can be blurred by a much more frequently congested link on the same path.

19. Conclusion
Pong – a tool specialized in measuring a subset of non-edge links exhibiting repetitive congestion
Coordinated probing
Infer underlying path topology conditions
Select probing techniques online
Quality of measurability
Quantify measurement quality
Link measurability score

20. Thank you!
Questions?