1. Link Failure Monitoring Using Network Coding
Hamed Firooz
Sumit Roy, Linda Bai

2. Outline Network Tomography Network Coding
Introduction (Network Monitoring)
Approaches:
Deterministic vs. Stochastic
Active vs Passive
Challenges: Overhead, Identifiability
Network Coding
Applications to network monitoring: new method
Optimization : speed/complexity tradeoffs
OPNET Implementation

3. Network Tomography
Networks: set of nodes, links modeled as graph G(V,E)
Network monitoring
Involves collection of network performance statistics (link delay, link loss or failure status)
Important for QoS guarantees (media streaming, interactive video applications)
Challenges
Choice of appropriate measurement technique and algorithmics 1 2 3 4 5 6 7 8
Node or network
A Logical Network
G(V,E)

4. Measurement Methods
Node-oriented: These methods are based on cooperation among network nodes, e.g. ping or traceroute
Using Ping, round trip delay to every node can be measured.
Uses Internet control message protocol (ICMP) packets
Many routers do NOT respond to these packets
Many service providers do not own the entire network l1 R l2 R R

5. Measurement Methods
Edge-oriented: Access is available to nodes at the edge only (and not to any in the interior)
Does not require exchanging special control messages between interior nodes
Inverse problem: estimate link level status from end-2-end (path level) measurements S S
Network(?) S S

6. Measurement Methods Passive (insitu traffic analysis)
Active (sending probe packets)
- Adds overhead to normal data traffic by
introducing new control packets
Passive (insitu traffic analysis)
- No overhead; temporal and spatial dependence might bias measurement
Our method: edge-oriented, active network tomography
Given a network, and a limited number of end hosts, when can we infer failure status of the links?

7. End-to-End Probing
Probes are inserted into a data stream, and end-to-end properties on that route measured.
Probes are exchanged between end nodes using routing matrix of the graph
End1
link1
router1
link2
link3
Routing matrix A
End2
End3

8. End-to-End Probes
Routing matrix relates link attribute to route attribute
For some parameters like delay or path loss, this relation is linear under some assumptions
End1 l1 R l2 l3
End2
End3

9. Deterministic
Link attributes (e.g. delay) are considered unknown, constant
Goal: estimate constants
Link attributes are typically time varying
 method is suitable for periods of local ‘stationarity’

10. Stochastic
Link attribute specified by a suitable probability distribution
e.g. link delay follows a Gaussian distribution
Estimation problem: unknown model parameters
based on path observation in the presence of additive noise

11. Deterministic vs. Stochastic Methods
Bayesian - requires a prior distribution
incorrect choice leads to biases in the estimates
More computationally intensive
Deterministic
Lower complexity but suffers from generic non-identifiability

12. Link Failure Model l1 l2 l3 R1 R2
End1
End2
Define an indicator function for status of each link

13. Binary Deterministic Model
End1
End2
y = Ax
A: N-by-M binary routing matrix
x: M-by-1 binary vector, the status of each link
y: N-by-1 binary vector, the status of each path (measurements)

14. Failure Monitoring Network G(V,E) with set of paths P
x, y are binary vectors
A path is congested if at least one of its links is congested
End1 l1
Router l2 l3
End2
End3

15. Identifiability y = Ax Problem: Estimate x from y with
A (N-by-M) : binary routing matrix
x (M-by-1) : binary link failure status
y (N-by-1) : end-to-end measurements
Identifiability: a network is identifiable if y = Ax has a unique solution
Usually, M ( # of links in network) >> N (# of measurements), so network is generically NOT identifiable.
6 links, 3 End-to-End routes  N=6, M=3

16. Identifiability: Binary Model
Solution: limit (maximum) number of failed links inside the network
Suppose at most k links can fail simultaneously
Defn: k-Identifiability
Network is k-identifiable if
from end-to-end observation it is possible to uniquely identify up to k congested links
Only one link can be congested

17. Example of 1-identifiability-- l1 l2 l3 l4 l5 l6 1 l2 l3 l5 l4 l6

18. Example: k=2 identifiability
Ambiguity l1 l2 l3 l5 l4 l6

19. 1-Identifiability
A network with an intermediate degree two node is not 1-identifiable
If path End1End2 is congested, it is impossible to determine which link among l1 and l2 is congested .
Necessary but not sufficient!

20. k=1 Identifiability P1 P3
1-identifiability Theorem: End-to-End probe based measurements can detect a unique congested link in a network if and only if there are no two identical columns in the network routing matrix P1 P3

21. k- identifiability
k-identifiability Theorem: End-to-End probe based measurements can detect a unique congested link in a network only if there are no k+1 dependent columns in the network routing matrix

22. Example: k=2 identifiability
Ambiguity l1 l2 l3 l5 l4 l6

23. Shortest Path Routing Revisited
Packets are sent on shortest path between two end nodes
- sub-graphs = tree starting from a boundary (source) node
Node 4 has degree two in all graphs
But node 4 has degree four in the original network

24. Revisiting Shortest Path Routing
What if we could change routing matrix ?
Example: in place of shortest path routing, route packets through longer paths, e.g. n1l2l4n2
Now network is 1-identifiable !
Intrinsic limitation for end-to-end measurement methods based on shortest path routes
probes transmitted along such paths contain only minimum information

25. Solution
Look to exchange probes between boundary nodes via other (non-shortest) paths?
Changing the routing tables violates tomography assumption
Use Network Coding; exploit broadcast nature of network coding, a transmitted probe will traverse almost every path between two boundary nodes

26. Network Coding: Short Review
Present: routers just forward incoming packets, i.e. copy the packets on an input link onto the output links
Proposed: What if each node in a network performs some computation on received data prior to forwarding? y1 y2 y3
f1(y1,y2,y3)
f2(y1,y2,y3) y1 y2

27. How does NC work? (1) receiver t2 A sender s D B receiver t1
“Butterfly” network: All links have the same capacity 1 b/s
s wants to send data bits a, b to both t1 and t2
Bottleneck is CD

28. How does NC work?(2) a receiver t2 A sender s a D b a+b B receiver t1
XOR b
a+b B
receiver t1 b
Node C XORs received messages on each of its links

29. How does NC work?(3) a receiver t2 A sender s a a+b D b a+b a+b B
XOR b
a+b
a+b B
receiver t1 b
t1 and t2 know both a and b
Now s can send data at rate 2 b/s/receiver

30. Linear Network Coding Network Coding is a coding at layer three
The coding is conducted over the finite field Fu, u=2q
each coded symbol can be represented by q-bits within an IP layer frame
Signal Y(j) on an outgoing link j of node v, is a linear combination of signals Y(i) on incoming link i of v:
We assume there is no process generated at node v

31. Received Symbols Pi : i-th route from source to destination
Source sends α over Pi
βi depends on topology G hence βi(G) α γ1 γ2 S γ3 D γ4 γ5

32. Received Symbols: Linear Model
ek one of source outgoing links
Pek : collection of all paths between source and destination starts at ek
Source sends αk over ek. By superposition destination receives γ1 α1 α γ2 e1 S γ3 D γ4 γ5

33. Received Symbols: Linear Model
Source sends out symbols αk over ek using superposition once more
In vector format: y=αtβ(G)
β(G) is total network coding vector γ1 α α1 γ2 e1 S γ3 D α2 γ4 γ5

34. Received Symbols: Linear Model
Source sends symbols in M succ. time slots:

35. Link Failure Model
If a link is severely congested, packets are significantly delayed and assumed lost at the destination
We model the network with link l in congestion state by its edge deleted subgraph denoted by Gl(V,El) γ1 S γ3 D γ4 γ5

36. Link Failure Model
Total network coding vecor of Gl(V;El), β(Gl) is different from β(G)
if the congested link doesn’t belong to i-th path from source to destination, Pi, it will not affect packets going through those paths
It is zero otherwise γ1 γ2 e1 l1 S γ3 D e2 γ4 γ5

37. Link Failure Model Training sequence is A
yl : vector of symbols observed at the destination in M time slots with link l congested
Potential for identifying: received symbols change uniquely in response to link congestion

38. Example -- e1 e2 l1 l2 l3 1st time slot 2 3 1 2nd time slot D S 1 1 e12 3 1
2nd time slot 1 1 e1 S 2 D e2 3 2

39. Theorem 1: Sufficient Conditions
If Rank(A)= deg(S), and
for all Pek set of paths between source and destination starting at ek
then
(more next slide)

40. Theorem 1 Condition means
For a set of paths having ek in common, Pek , NC coefficient of the paths are independent !
Independent
Independent γ1 γ2 e1 S γ3 D e2 γ4 γ5

41. Example -- e1 e2 l1 l2 l3 1st time slot 2 3 1 2nd time slot D S 1 1 e1
Independent -- e1 e2 l1 l2 l3
1st time slot 2 3 1
2nd time slot 1 1 e1 S 2 D e2 3 2

42. Complexity/Speed First condition of Theorem 1:
In previous example M=2=deg(S)
Number of time slots: at least the number of outgoing links of source
Is it possible to decrease number of time slots?  faster monitoring
Possible by increasing number of bits in LNC coeff.  more complexity

43. Example q=3 A=[1 1 4] -- e1 e2 l1 l2 l3 1st time slot 6 4 2 5 7 1 D S

44. Theorem 2: Complexity/Speed tradeoff
Ni=|Pi|
q bits per symbol are used in network coding
M number of (desired) time slots
Let Z={1,2,…,K}
K degree of source
ZM: collection of all partitions of Z with size M
K=3, 2  Z={1,2,3}
ZM={ {{1,2},{3}} , {{1,3},{2}} , {{2,3},{3}} }
K links S

45. Theorem 2: Complexity/speed tradeoff
Network is 1-identifiable if
Rank(A)=M

46. Theorem 3: Random LNC
Random linear network coding is a distributed approach achieving capacity asymptotically
Intermediate node choose their NC coefficients uniformly from the elements of Fu (u=2q)
Exponential increase with q (number of bits) and M (number of time slots)
Quadratic decrease with size of network

47. Multi-source Multi-destination
So far, considered only Single source Single destination
Easily extendable to Multi-source Multi-destination

48. Simulation Simulation environment Evaluation OPNET 14.5
MATLAB 7.1 (finite field operations)
Evaluation
University of Washington’s Electrical Engineering network
Thirteen subnets
3 backbone routers
Full Duplex Ethernet links

49. Simulation Set-Up Implementation of Network Coding (NC) within OPNET
We employ network coding at transport layer (instead of IP layer)
Easier to implement
Routers model is modified to distinguish between non-NC/NC packets through the use of a flag bit within the UDP header
NC packets are sent for separate processing
non-NC packets are processed normally
We assign a q-bit field called LNC field within the TCP/UDP header, for linear network coding.
UDP packet
LNC field 1

50. RECEIVE/SEND interface
Inherently network coding operates on unidirectional links
Each interface within a router mode is designated as a SEND or RECEIVE interface only for the network coded packets
operating regularly with non-network coded packets
Finite field operation is done in MATLAB
Using MATLAB API within OPNET

51. RECEIVE/SEND

52. Evaluation

53. UW EE Network

54. UW EE Network-lookup table

55. Thank you

56. Network Tomography: A Stochastic Model [1]
Passage of probes can be modeled as two stochastic process: {Xl(i)} and {Zl(i)} for each node k
Zl(i) time delay process of link k
Xl(i) called bookkeeping process: cumulative probe from root to k
[1] V. Arya, N. Duffield, D. Veitch “ Temporal Delay Tomography”, IEEE Infocom 2008

57. Network Tomography: Stochastic Methodl
Xl(i)
Zl(i) l’
Xl’(i)
Discretize delay D={0,b,2b,…,mb,∞}
mb is delay threshold
Xl(i)=Xl’(i)+Zl(i)