1. 1 A General Introduction to Tomography & Link Delay Inference with EM Algorithm Presented by Joe, Wenjie Jiang 21/02/2004

2. 2 Outline of Talk Why tomography? Introduction to tomography Internal Link Delay Inference Basic EM A simple example to infer internal link delay using EM algorithm Conclusion

3. 3 Terminology “ Tomography ” Brain Tomography Access is difficult! Network Tomography Access is difficult! Vardi 1996

4. 4 Why tomography? What is the: Bandwidth? Loss rate? Link Delay? Traffic demands? Connectivity of links in the network? (Topology Inference) Path: a connection between two end nodes, each consisting of several links. Link: a direct connection with no intermediate routes/hosts.

5. 5 Motivation Identify congestion points and performance bottlenecks Dynamic routing Optimized service providing Security: detection of anomalous/malicious behavior Capacity planning

6. 6 Why tomography - Difficulty Decentralized, heterogeneous and unregulated nature of the internal network. No incentive for individuals to collect and distribute these info freely. Collecting all statistics impose an impracticable overhead expense ISP regards the statistics highly confidential Relaying measurements to decision-making point consumes bandwidth.

7. 7 Why tomography - Solution Widespread internal network monitoring is expensive and infeasible Edge-based measurement and statistical analysis is practical and scalable

8. 8 Brain Tomography

9. 9 Network Tomography

10. 10 Where are you? Why tomography? Introduction to tomography Internal Link Delay Inference Basic EM A simple example to infer internal link delay using EM algorithm Conclusion

11. 11 Introduction to tomography Use a limited number of measurements to infer network (link) performance parameters, using: -- Maximum Likelihood Estimator -- Estimation Maximization -- Bayesian Inference and assuming a prior model. Categories of problems: -- Link level parameter estimation -- Sender-Receiver traffic intensity. -- Topology Inference

12. 12 Introduction to tomography (2) Two forms of network tomography: -- link-level metric estimation based on end-to- end, traffic measurements (counts of sent/received packets, time delays between sent/received packets) -- path level (sender-receiver path) traffic intensity estimation based on link-level measurements (counts of packets through nodes) Passive or Active measurements? Multicast or Unicast?

13. 13 Problem Description To solve the linear system: A, ө and εhave special structures. Goal: to maximize the likelihood function

14. 14 Problem Description (2) A = routing matrix (graph) ө = packet queuing delays for each link y = packet delays measured at the edge ε= noise, inherent randomness in traffic measurements Statistical likelihood function

15. 15 Problem Description (3) An virtual multicast tree with four receivers l1l1 l2l2 l3l3 l4l4 l5l5 l6l6 l7l7 l1l1 l2l2 l3l3 l4l4 l5l5 l6l6 l7l7 Y1Y2Y3Y4 Y1=X1+X2+X4

16. 16 Where are you? Why tomography? Introduction to tomography Internal Link Delay Inference Basic EM A simple example to infer internal link delay using EM algorithm Conclusion

17. 17 Physical Topology Measure end-to-end (from sender to receiver) delays

18. 18 Logical Topology Logical topology is formed by considering only the branching points in the physical topology Infer the logical link-level queuing delay distributions!

19. 19 The basic idea of internal link delay tomography Send a back-to-back packet pair from a sender, each packet heading to a different receiver Use the fact that delays are highly correlated on shared links Queuing delay difference between these two end can be attributed to the unshared links

20. 20 Delay Estimation Measure end-to-end delay of packet pairs Packets experience the same delay on link1 d 2 =d min =0d 3 >0Extra delay on link 3!

21. 21 Packet-pair measurements Key Assumptions Fixed known routes Temporal independence Spatial independence Packet-pair delays are identical on share links. N delay measurements in all

22. 22 Parameters α1α1 α2α2 α3α3 α4α4 α5α5 α6α6 α7α7 α8α8 α9α9 α i = parameter of delay pmf on link i

23. 23 Link delay model α i = delay pmf on link i Link delay model could be multinomial quantized delay model: delay= {0, 1, 2, 3,…,L,∞} α i= {α i0,α i1,α i2,...,α iL,α i ∞ } α ij =P{ delay(link i) = j } α i0 +α i1 +α i2,...,α iL +α i ∞= 1

24. 24 Goal is the probability of the event of n-th measurement is the probability of the event of all measurements Our goal: find

25. 25 Where are you? Why tomography? Introduction to tomography Internal Link Delay Inference Basic EM A simple example to infer internal link delay using EM algorithm Conclusion

26. 26 Review of MLE (Maximum Likelihood Estimation)

27. 27 Review of MLE (Maximum Likelihood Estimation) The basic idea of MLE: God always let the event with the biggest probability happen the most likely -- The MLE of ө is to make the sample occur the most likely Note we assume X={x 1,…x N } to be i.i.d The solution could be easy or hard depending on the form of p(ө|X) e.g. p(ө|X) is a single Gaussian ө=(μ, σ 2 ), we can set the derivative of logL(ө|X) to zero and solve it directly.

28. 28 Complete Data The sample X={x 1,…x N } together with the missing (or latent) data Y is called complete data. The complete likelihood is where p(x, y|ө) is the joint density of X and Y given the parameter ө. The complete log-likelihood is

29. 29 Complete MLE By the definition of conditional density, where p(y|x,ө) is the conditional density of Y given X=x and ө The complete MLE

30. 30 Basic idea of EM Given X=x and ө= ө t-1, where ө t-1 is the current estimates the unknown parameters log p(x,Y| ө) is a function of Y whose unique best Mean Squared Error (MSE) predicator is

31. 31 EM steps

32. 32 The magic of EM the direct MLE of is relatively hard to solve But the MLE of complete log-likelihood is relatively easier to obtain since is a function of x and y, (y is hidden), we use the expectation of y under x and So E-step M-step

33. 33 Where are you? Why tomography? Introduction to tomography Internal Link Delay Inference Basic EM A simple example to infer internal link delay using EM algorithm Conclusion

34. 34 EM in link delay inference α1α1 α2α2 α3α3 α4α4 α5α5 α6α6 α7α7 α8α8 α9α9 x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 x8x8 x9x9 Note that here notation x and y have opposite meaning of x, y stated in previous EM algorithm

35. 35 EM in link delay inference (2) Complete data Z=(X,Y) the complete data log-likelihood: P α [Y|X] has nothing to do with α m i,j is the total number of packets experience a delay j on link i over N measurements.

36. 36 EM in link delay inference (3) The MLE of αwould be

37. 37 EM in link delay inference (4) MLE which is the frequency of event m i A simple example is that we toss a die, P( the result i)=α i (i=1,2…6) m i = how many times we see result i

38. 38 EM in link delay inference (5) We notice that is similar to only different that should be replaced by So the MLE

39. 39 EM in link delay inference (6) Probability Propagation

40. 40 A simple example delay on each link fall into {0,1,2,3} y1 y2 x1 x2x3 0 1 23 α ij =P{ delay (link i) = j }

41. 41 A simple example (2) Suppose there are 5 measurements: { (3,2), (4,2), (6,5), (0,0), (4,1)} y1 y2 x1 x2x3 0 1 23

42. 42 A simple example (3) y1 y2 x1 x2x3 0 1 23 Bayes Formula

43. 43 A simple example (4) y1 y2 x1 x2x3 0 1 23

44. 44 A simple example (5) y1 y2 x1 x2x3 0 1 23 similarly:

45. 45 A simple example (6) j i 0123 14/311/65/61 211/35/617/6 3 5/64/30 m i,j computed in the first iteration.

46. 46 A simple example (7) the physical meaning of α 1,0 is that: the number of packets that experience delay 0 on link i divided by the total number of packets that travel through link i

47. 47 A simple example (8) j i 0123 14/1511/301/61/5 2 1/151/617/30 3 1/64/150 α i,j computed in the first iteration

48. 48 A simple example (9) j i 0123 10.4 00.2 2 000.8 30.40.20.40 Iteration: iterate E-step and M-step, until some termination criteria is satisfied! After 6 iterations, α i,j converges to a fixed value.

49. 49 A simple example (9) { (3,2), (4,2), (6,5), (0,0), (4,1)} y1 y2 x1 x2x3 0 1 23

50. 50 Complexity

51. 51 Where are you? Why tomography? Introduction to tomography Internal Link Delay Inference Basic EM A simple example to infer internal link delay using EM algorithm Conclusion

52. 52 Conclusion + The field is just emerging. Deploying measurement/probing schemes and inference algorithms in larger networks is the next key step.

53. 53 Problems The spatial-temporally stationary and independent traffic model has limitations, especially in heavily loaded networks. A trend for highly uncooperative environment for active probing – passive traffic monitoring techniques, for example based on sampling TCP traffic streams

54. 54 Thank you! The End