1. Analysis of the Impact and Interactions of Protocol and Environmental Parameters on Overall MANET Performance Michael W. Totaro and Dmitri D. Perkins Center for Advanced Computer Studies University of Louisiana at Lafayette Presented by Michael W. Totaro

2. Topics  Introduction and Motivation  2 k Factorial Design (A Brief Tutorial)  Related Work  Methodology  Analysis, Results, and Models  Future Work and Open Questions  Questions  References

3. Topics  Introduction and Motivation  2 k Factorial Design (A Brief Tutorial)  Related Work  Methodology  Analysis, Results, and Models  Future Work and Open Questions  Questions  References

4. Introduction and Motivation  Important design challenge of MANETs: maximize overall performance of protocols operating in a MANET  Impact that one or more factors have on MANET performance?  2 k factorial design—an important tool that may aid researchers in analyzing effect of factors on MANET performance

5. Topics  Introduction and Motivation  2 k Factorial Design (A Brief Tutorial)  Related Work  Methodology  Analysis, Results, and Models  Future Work and Open Questions  Questions  References

6. 2 k Factorial Design  Example of a design that is feasible in many simulations: 2 k factorial design  Have k factors (inputs), each at just two levels  Number of possible combinations of factors—usually called design points—is 2 k

7. 2 k Factorial Design Process  Code each factor to a “+” and a “-” level  Design matrix: All possible combinations of factor levels  Example for k = 3 factors: Make the 8 simulation runs, and measure the effects of the factors!

8. 2 k Factorial Design Main Effect of a Factor Main effect of a factor is the average difference in the response when this factor is at its “+” level as opposed to its “-” level:

9. 2 k Factorial Design Main Effect of a Factor – cont’d The main effects measure the average change in the response due to a change in an individual factor, with this average being taken over all possible combinations of the other k-1 factors (numbering 2 k-1 ).

10. 2 k Factorial Design Main Effect of a Factor – cont’d We can rewrite the above as “Factor 1” column ● “Response” column / 2 k-1 -R 1 + R 2 – R 3 + R 4 – R 5 + R 6 – R 7 + R 8 e 1 = 4

11. 2 k Factorial Design Factor Interaction  Two factors A and B are said to interact if the effect of one depends upon the level of the other  Conversely, these two factors, A and B, are said to be noninteracting if the performance of one is not affected by the level of the other  We shall look at examples of interacting factors and noninteracting factors

12. 2 k Factorial Design Examples of Noninteracting and Interacting Factors A1A1 A2A2 B1B1 35 B2B2 68 Noninteracting Factors Interacting Factors A1A1 A2A2 B1B1 35 B2B2 69 As the factor A is changed from level A 1 to level A 2, the performance increases by 2 regardless of the level of factor B As the factor A is changed from level A 1 to level A 2, the performance increases either by 2 or 3 depending upon whether B is at level B 1 or level B 2, respectively

13. 2 k Factorial Design Examples of Noninteracting and Interacting Factors – cont’d Performance Graphical representation of interacting and noninteracting factors. 6 2 8 A1A1 A2A2 B2B2 B1B1 Performance 6 2 8 B1B1 A2A2 A1A1 B2B2 (a) No Interaction Performance 6 2 8 A1A1 A2A2 B2B2 B1B1 6 2 8 B1B1 A2A2 A1A1 B2B2 (b) Interaction

14. 2 k Factorial Design Interaction Effects 1 x 3 interaction effect: “Factor 1” ● “Factor 3” ● “Response” / 2 k-1 R 1 - R 2 + R 3 - R 4 – R 5 + R 6 – R 7 + R 8 e 13 = 4  Addresses the question: “Does the effect of a factor depend on level of others?”  Sign of effect indicates direction of effect on response of moving that factor from its “-” to its “+” level

15. Topics  Introduction and Motivation  2 k Factorial Design (A Brief Tutorial)  Related Work  Methodology  Analysis, Results, and Models  Future Work and Open Questions  Questions  References

16. Related Work  Interest in cross-layer factor interaction in MANETs is not entirely new.  Performance metrics for assessing the behavior of MANETs are identified, discussed, and, most especially, partitioned into three classification levels [1]  Thread-task level metrics (algorithmic level) such as average power expended and task completion time  Diagnostic packet level metrics such as end-to-end throughput, end-to-end delay, link utilization, and packet loss, which characterize network behavior at the packet level  Scenario metrics that describe the network environment and define the scenario; these include: nodal movement/topology rate of change, number of network nodes, area size of network, density of nodes per unit area, offered load and traffic patterns, and number of unidirectional links

17. Related Work – cont’d  A comprehensive analysis of five factors—node speed, pause-time, network size, number of traffic sources, and type of routing (source vs. distributed)—was done using a factorial experimental design in an effort to identify and quantify the effects and two-way interactions of these factors on three performance responses: throughput, average routing overhead, and power consumption [2]  Potential benefits may be derived by information exchange between the lower layer, routing layer, and transport layer, which is useful in the design and standardization of an adaptive architecture that can exploit the interdependencies among link, medium access, network, and applications protocols [3]  The underlying premise in cross-layer interaction analyses is that, by learning more about factor and two-way interactions on the performance of MANETs, researchers may want to consider taking these effects into account in the design of future protocols

18. Topics  Introduction and Motivation  2 k Factorial Design (A Brief Tutorial)  Related Work  Methodology  Analysis, Results, and Models  Future Work and Open Questions  Questions  References

19. Methodology Effects of factors?

20. Methodology – cont’d Partial Design Grid (Coded)

21. Methodology – cont’d Partial Design Grid (Uncoded)

22. Methodology – cont’d  Simulation—QualNet 2 6 factors = 64 experimental runs Replications = 5 Total running time = 320 seconds LAR1 routing protocol Free-space model 2 Mbps Packet size = 512 bytes Pause time = 25 seconds Transmission range = 250 meters Terrain dimensions * : N = √ [ (MR 2 π / X) – 1 ] where X = average number of neighbors M = number of nodes R 2 = transmission range *Using the formula for computing the average number of neighbors for a node, derived by Ihklas Ajbar.

23. Topics  Introduction and Motivation  2 k Factorial Design (A Brief Tutorial)  Related Work  Methodology  Analysis, Results, and Models  Future Work and Open Questions  Questions  References

24. Analysis, Results, and Models Scatterplot—Packet delivery ratio

25. Analysis, Results, and Models Scatterplot—End-to-end delay

26. Analysis, Results, and Models Scatterplot—Control packet overhead

27. Analysis, Results, and Models Main effects—Packet delivery ratio

28. Analysis, Results, and Models – cont’d Main effects—End-to-end delay

29. Analysis, Results, and Models – cont’d Main effects—Control packet overhead

30. Analysis, Results, and Models – cont’d Two-way factor interactions—Packet delivery ratio

31. Analysis, Results, and Models – cont’d Two-way factor interactions—Control packet overhead

32. Analysis, Results, and Models – cont’d Two-way factor interactions—End-to-end delay

33. Analysis, Results, and Models – cont’d Response-surface plots

34. Analysis, Results, and Models – cont’d Contour plots

35. Analysis, Results, and Models – cont’d Predictive effects—Packet delivery ratio

36. Analysis, Results, and Models – cont’d Predictive effects—End-to-end delay

37. Analysis, Results, and Models – cont’d Predictive effects—Control packet overhead

38. Analysis, Results, and Models – cont’d Prediction profile

39. Analysis, Results, and Models – cont’d Regression Models—Packet delay ratio

40. Analysis, Results, and Models – cont’d Regression Models—End-to-end delay

41. Analysis, Results, and Models – cont’d Regression Models—Control packet overhead

42. Topics  Introduction and Motivation  2 k Factorial Design (A Brief Tutorial)  Related Work  Methodology  Analysis, Results, and Models  Future Work and Open Questions  Questions  References

43. Future Work and Open Questions  Validation of prediction (regressions) models  Other factors (e.g., power consumption, etc.)?  Other empirical models (e.g., neural networks, time-series prediction)?

44. Topics  Introduction and Motivation  2 k Factorial Design (A Brief Tutorial)  Related Work  Methodology  Analysis, Results, and Models  Future Work and Open Questions  Questions  References

45. Questions  Thank you!

46. Topics  Introduction and Motivation  2 k Factorial Design (A Brief Tutorial)  Related Work  Methodology  Analysis, Results, and Models  Future Work and Open Questions  Questions  References

47. References 1.M. W. Subbarao, “Ad Hoc Networking Critical Features and Performance Metrics”. White paper, Wireless Communications Technology Group, National Institutes of Standards and Technology, September 15, 1999 2.D. D. Perkins, H. D. Hughes, and C. B. Owen, “Factors Affecting the Performance of Ad Hoc Networks”. Proceedings of IEEE International Conference on Communications (ICC 2002), New York, April 2002 3.J. Lee, S. Singh, and Y. Roh, “Interlayer Interactions and Performance in Wireless Ad Hoc Networks,” Internet – Draft, IRTF ANS Working Group; http://www.flarion.com/ans-research/Drafts/draft- irtf-ans-interlayer-performance-00.txt, accessed 1/29/2004 http://www.flarion.com/ans-research/Drafts/draft- irtf-ans-interlayer-performance-00.txt