1. Avoiding traceroute anomalies with Paris Traceroute
Brice Augustin
(Université Pierre et Marie Curie-LIP6/CNRS)
Xavier Cuvellier, Timur Friedman, Renata Teixeira
Fabien Viger, Benjamin Orgogozo, Matthieu Latapy
(Université Denis Diderot-LIAFA/CNRS)
Clémence Magnien
(Ecole Polytechnique-CREA/CNRS)

2. Contributions
Identified traceroute deficiencies on load balanced paths
Measured paths are inaccurate and incomplete
False diamonds, false loops and false cycles
Built a new traceroute: Paris traceroute

3. Traceroute under load balancing
Actual topology: A C E
Dst
Src L
TTL = 2 B D
TTL = 3
Inferred topology: A C
False link E
Dst
Src L B D
Missing nodes and links

4. Anomalies: false diamondsC
Actual topology: E
Dst
Src L B D
Inferred topology: A C E
Dst
Src L B D

5. Anomalies: false loops and cycles
TTL = 3
Actual topology: A
Src L D
Dst B C
TTL = 4
TTL = 2
Inferred topology:
Natural under per-packet, but we found that per-packet is rare. Why does it happen under per flow.
Src L D
Dst B

6. Anomalies happen even under per-flow load balancing
Port 2
Flow 1 A C
Port 1
Dst
Src E L
TTL = 2
Port 3 B D
TTL = 1
TTL = 3
We create a new tool that solves this problem: keeps the flow id constant
Traceroute uses the destination port as identifier
Per-flow load balancers use the destination port as part of the flow identifier

7. Paris traceroute Solves the problem with per-flow load balancing
Probes to a destination belong to same flow
How to identify probes?
Use the UDP checksum
Does not address per-packet load balancing
Checksum 2
Checksum 3
Port 1
Port 1 A C
Checksum 1
Port 1 E
Dst
Src L
TTL = 2
TTL = 3 B D
TTL = 1

8. Measurement infrastructure
5000 reachable destinations
Measurements
1 round takes 100 minutes
2 months
1465 rounds
Paris traceroute
Classic traceroute
Paris traceroute
Classic traceroute
Paris
Paris traceroute
INTERNET
Source

9. Unusual observations Diamonds Loops Cycles
“good” causes of diamonds, loops, and cycles: routing dynamics
Might be caused by legitimate routing dynamics, but we found that are often caused by measurement artifacts. D C C
What portion of these are false, i.e. anomalies?

10. From our LIP6 vantage point:
Measurement artifacts are common
From our LIP6 vantage point:
Diamonds appear in 30% of the destinations
Paris traceroute removes 10,662 from 19,159 (56%)
Loops appear in 4.5% of the measured routes
Paris traceroute removes 5,047 from 5,795 (87%)
Cycles appear in 0.25% of the measured routes
Paris traceroute removes 3,886 from 5,674 (68%)
Other causes
Routing changes
NAT boxes
Buggy routers
Per-packet load balancing

11. Conclusion Classic traceroute leads to anomalies:
False diamonds, false loops and false cycles
Per-flow load balancers cause most of the anomalies
Paris traceroute reports more precise paths

12. More information

13. Future Directions Exhaustive algorithm Broader experiments
Find all paths
Fine characterization of load balancers
Impact on inferred graphs (stats, dynamics)
Broader experiments
More sources and destinations
Path diversity
Optimize Internet connections by selecting the appropriate flow identifier

14. Anomalies: false loops and cycles
TTL = 3 A
Dst
Src D L B C E
TTL = 5
TTL = 2
TTL = 4 D
Dst
Src L B E

15. Anomalies: Loops caused by buggy routers
-bash$ traceroute Dst
traceroute to Dst
1 B ms
2 B ms
3 Dst ms
Src A B
Dst X
Rejects the probe with a TTL of 0 and sends it back to the source
Forwards the probe with TTL equal to 0
Forwards the probe with TTL equal to 0
TTL = 1
Rejects the probe with a TTL of 0 and sends it back to the source
TTL = 1
TTL = 2
-bash$ traceroute-paris Dst
traceroute to Dst
1 B ms !T0
2 B ms
3 Dst ms
Src B
Dst

16. Anomalies: Loops caused by NAT boxes
Response TTL = 254
IP Identifier = 12375
Response TTL = 252
IP Identifier = 9356
Src
Dst
(NAT) A B C
Dst B
TTL = 2
TTL = 3
TTL = 3
Response TTL = 253
IP Identifier = 5286 2
Src A
Dst
See [ Bellovin 2002 ]

17. An intriguing traceroute output
-bash$
-bash$ traceroute F
traceroute to F, 64 hops max, ...
1 A ms ms ms
2 B ms C ms B ms
3 D ms E ms D ms
4 F ms ms ms A B C ? D E F

18. An intriguing traceroute output
-bash$
-bash$ traceroute F
traceroute to F, 64 hops max, ...
1 A ms ms ms
2 B ms C ms B ms
3 D ms E ms D ms
4 F ms ms ms
1 A ms ms ms
2 C ms ms B ms
3 E ms D ms ms
4 F ms ms ms A B C ? D E F

19. Anomalies: diamonds A C E
Dst
Src L B D A C
Dst
Src E L B D

20. What we expect with per-flow load balancing
Dst
Src E L
TTL = 2 B D
TTL = 3 A
Dst
Src E L D

21. How traceroute works Src Dst A B Src Dst A0 B0 Src Dst A1 B1 TTL = 1A 1 B 1
TTL = 1
TTL = 2
TTL = 3
Src
Dst A0 B0
Src
Dst A1 B1