1. The Design of XML-Based Model and Experiment Description Languages for Network Simulation Andrew Hallagan Bucknell University Dept. of Computer Science Luiz Felipe Perrone, Advisor April 26, 2011 1

2. 3 General Network Types Clockwise from top: hardwired, wireless, and ad-hoc networks 2

3. 3 General Network Types Clockwise from top: hardwired, wireless, and ad-hoc networks Nodes 3

4. 3 General Network Types Clockwise from top: hardwired, wireless, and ad-hoc networks Channels 4

5. Communication Between Nodes Breaking a message up into packets. 5

6. Communication Between Nodes Breaking a message up into packets. 6

7. Communication Between Nodes Breaking a message up into packets. 7

8. Ways to Study a System Source: Averill M. Law, Simulation, Modeling & Analysis Physical model Experiment with the actual system Experiment with a model of the system System Mathematical model Analytical solution Simulation 8

9. Physical model Experiment with the actual system Experiment with a model of the system System Mathematical model Analytical solution Simulation Ways to Study a System Source: Averill M. Law, Simulation, Modeling & Analysis Costly, disruptive 9

10. Physical model Experiment with the actual system Experiment with a model of the system System Mathematical model Analytical solution Simulation Ways to Study a System Source: Averill M. Law, Simulation, Modeling & Analysis Costly, disruptive Not applicable 10

11. Physical model Experiment with the actual system Experiment with a model of the system System Mathematical model Analytical solution Simulation Ways to Study a System Source: Averill M. Law, Simulation, Modeling & Analysis Costly, disruptive Not applicable d = v∙t 11

12. Physical model Experiment with the actual system Experiment with a model of the system System Mathematical model Analytical solution Simulation Ways to Study a System Source: Averill M. Law, Simulation, Modeling & Analysis Costly, disruptive Not applicable d = v∙t 12

13. Physical model Experiment with the actual system Experiment with a model of the system System Mathematical model Analytical solution Simulation Ways to Study a System Source: Averill M. Law, Simulation, Modeling & Analysis Costly, disruptive Not applicable Difficult, if not impossible 13

14. Plotting a regression equation Regression equations are a form of metamodel that researchers use to predict relationships in a model that cannot be derived analytically. time distance 14

15. A Simple Mobility Model The horizontal and vertical positions of the node are a function of SPEED and ANGLE, which are random variables distributed Normally and Uniformly, respectively. X-position = f(SPEED, ANGLE) Y-position = g(SPEED, ANGLE) SPEED ~ N(SPEED_MEAN, 3.0) ANGLE ~ U(0, ANGLE_BOUND) Model Inputs SPEED_MEAN ANGLE_BOUND 15

16. A Simple Mobility Model The horizontal and vertical positions of the node are a function of SPEED and ANGLE, which are random variables distributed Normally and Uniformly, respectively. X-position = f(SPEED, ANGLE) Y-position = g(SPEED, ANGLE) SPEED ~ N(SPEED_MEAN, 3.0) ANGLE ~ U(0, ANGLE_BOUND) Model Inputs SPEED_MEAN ANGLE_BOUND Model Description 16

17. Factor Levels ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 17

18. Design Matrix ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 18

19. Design Matrix ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 design point 19

20. Design Matrix ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 20

21. Design Matrix ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 Experiment Description 21

22. Network Complexities Bandwidth, physical distance, network traffic, etc. 22

23. Network Complexities Bandwidth, physical distance, network traffic, etc. 23

24. Network Complexities Scale. 24

25. Network Complexities Physical surroundings. 25

26. Various simulator engines Researchers have a number of simulator engines to choose from, including ns-2, ns-3, OPNET, CSIM, GloMoSim and MATLAB. 26

27. Various simulator engines Researchers have a number of simulator engines to choose from, including ns-2, ns-3, OPNET, CSIM, GloMoSim and MATLAB. ns-2 27

28. Various simulator engines Researchers have a number of simulator engines to choose from, including ns-2, ns-3, OPNET, CSIM, GloMoSim and MATLAB. ns-2 ns-3 28

29. Various simulator engines Researchers have a number of simulator engines to choose from, including ns-2, ns-3, OPNET, CSIM, GloMoSim and MATLAB. ns-2 ns-3 29

30. Various simulator engines Researchers have a number of simulator engines to choose from, including ns-2, ns-3, OPNET, CSIM, GloMoSim and MATLAB. ns-2 ns-3 GloMoSim 30

31. Various simulator engines Researchers have a number of simulator engines to choose from, including ns-2, ns-3, OPNET, CSIM, GloMoSim and MATLAB. ns-2 ns-3 CSIM GloMoSim 31

32. Various simulator engines Researchers have a number of simulator engines to choose from, including ns-2, ns-3, OPNET, CSIM, GloMoSim and MATLAB. ns-2 ns-3 CSIM GloMoSim SWAN 32

33. Various simulator engines Researchers have a number of simulator engines to choose from, including ns-2, ns-3, OPNET, CSIM, GloMoSim and MATLAB. ns-2 ns-3 CSIM GloMoSim M ATLAB SWAN 33

34. Recap Networks are complex. Modeling is difficult. Experimentation isn’t straight-forward. Mistakes in this process have led to credibility issues in network simulation. 34

35. Solution: Automation Automation provides guidance that helps increase credibility of published research. Automate the modeling and experiment design processes. Validate experiments at multiple levels. 35

36. Motivation for Network Simulation Research Source: J. M. Brase and D. L. Brown, Modeling, Simulation and Analysis of Complex Networked Systems. White paper. “[The Department of Energy] must continue the development of next generation complex networked systems that are more secure, less brittle to unexpected events, and more controllable. For these emerging efforts to be successful, it is essential that a firm intellectual foundation be provided for understanding and simulating large-scale networks.” 36

37. SAFE Simulation Automation Framework for Experiments 37

38. Various simulator engines Researchers have a number of simulator engines to choose from, including ns-2, ns-3, OPNET, CSIM, GloMoSim and MATLAB. ns-2 CSIM GloMoSim M ATLAB SWAN ns-3 38

39. High-level view of the SAFE architecture. My work is concerned with the Model Description and Experiment Description inputs to this framework (in the yellow boxes). 39

40. High-level view of the SAFE architecture. My work is concerned with the Model Description and Experiment Description inputs to this framework (in the yellow boxes). 40

41. An HTML “anchor” element. This piece of HTML is valid XML. It is an a tag with an attribute href. The value of the href attribute is a URL string and the content of the a tag is the “Link to Google” string. Link to Google Opening tag Attribute value Element content Closing tag 41 Namespace prefix

42. An HTML “table” element. This piece of HTML is valid XML. It is a table tag with nested tr, th, and td elements. School State Bucknell University PA 42

43. Related Work 43

44. Related Work Kurkowski et. al. (2005) examined 114 published studies. Perrone, Cicconetti, Stea (2009) argue for automation. Rioux et. al. (2008) talk about benefits of using XML across whole pipeline. Röhl et. al. (2006) argue XML enables fast development of UIs. Canonico et. al. (2003) used XML for scenario description, code generation. 44

45. NEDL: Experiment Description 45

46. NEDL Functionality Provide ways to list experimental factors. ANGLE_BOUNDSPEED_MEAN 46

47. A NEDL “factorlist” element The “factorlist” element lists all of the factor names referenced throughout the rest of the NEDL document. SPEED_MEAN ANGLE_BOUND 47

48. NEDL Functionality Provide ways to list associated level values for each factor. ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 48

49. NEDL Functionality Provide ways to list associated level values for each factor. ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 “Use the values 3, 17, and 9.” 49

50. A NEDL “memberof” element The “memberof” element associates a factor with a list of level values. The name was inspired by the terminology used to describe mathematical sets. SPEED_MEAN speed-values 3 17 9 50

51. A NEDL “memberof” element The “memberof” element associates a factor with a list of level values. The name was inspired by the terminology used to describe mathematical sets. SPEED_MEAN speed-values 3 17 9 Can be re-used; can be an external filename. 51

52. NEDL Functionality Provide ways to list associated level values for each factor. ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 “Use the values specified in my external file named ____” 52

53. NEDL Functionality Provide ways to list associated level values for each factor. ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 “Use the same values already provided for another factor.” " 53

54. A NEDL “memberof” element The “memberof” element associates a factor with a list of level values. The name was inspired by the terminology used to describe mathematical sets. SPEED_MEAN speed-values ANOTHER_FACTOR speed-values 3 17 9 54

55. NEDL Functionality Provide ways to list associated level values for each factor. ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 “Use values of the form 5∙x + 2, where 0 ≤ x ≤ 9” 55

56. A NEDL “sequence” element The “sequence” element associates a factor with a sequence of level values. The sequence expression is built up using constants, variables, and further expression elements. SPEED_MEAN EQUALS 2 PLUS 5 MULT x x 0 9 1 56

57. NEDL Functionality Provide ways to “prune” design points from the experiment space. ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 57

58. NEDL Functionality Provide ways to “prune” design points from the experiment space. ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 “Remove the specific design point where ANGLE_BOUND = 15 and SPEED_MEAN = 3.0.” 58

59. A NEDL “exclusionrestriction” element The “exclusionrestriction” element specifies a whole or partial design point which should not be included in the final experiment space. 59

60. NEDL Functionality Provide ways to “prune” design points from the experiment space. ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 “Remove any design points where SPEED_MEAN = 4.0.” 60

61. A NEDL “exclusionrestriction” element The “exclusionrestriction” element specifies a whole or partial design point which should not be included in the final experiment space. 61

62. NEDL Functionality Provide ways to “prune” design points from the experiment space. ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 “Link the factors ANGLE_BOUND and SPEED_MEAN.” 62

63. NEDL Functionality Provide ways to “prune” design points from the experiment space. ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 2.0 3.0 4.0 “Link the factors ANGLE_BOUND and SPEED_MEAN.” ANGLE_BOUND 15 30 45 60 SPEED_MEAN 2.0 3.0 4.0 63

64. A NEDL “linkingrestriction” element The “linkingrestriction” element specifies factors which should be “linked” when creating design points. SPEED_MEAN ANGLE_BOUND 64

65. NEDL Validation The NEDL validation module checks language semantics and ensures the experimenter has created a NEDL document that can be successfully parsed by SAFE. 65

66. NSTL: Model Description 66

67. Creating an ns-3 script template. One can create an ns-3 script simply by removing certain hard- coded values and inserting special markers that correspond to the factors listed in a NEDL file. // A bunch of ns-3 code before…. norm = ns3.RandomVariable(“Normal”, 2.0, 3.0) unif = ns3.RandomVariable(“Uniform”, 0, 90) node.setSpeed(norm) node.setAngle(unif) // A bunch of ns-3 code after… 67

68. Creating an ns-3 script template. One can create an ns-3 script simply by removing certain hard- coded values and inserting special markers that correspond to the factors listed in a NEDL file. // A bunch of ns-3 code before…. norm = ns3.RandomVariable(“Normal”, $SPEED_MEAN$, 3.0) unif = ns3.RandomVariable(“Uniform”, 0, $ANGLE_BOUND$) node.setSpeed(norm) node.setAngle(unif) // A bunch of ns-3 code after… 68

69. Creating an ns-3 script template. Using the block element in NSTL, one can specify interchangeable code blocks to swap in an ns-3 script. Since two block elements are used, two scripts will be generated. // A bunch of ns-3 code before… norm = ns3.RandomVariable(“Normal”, $SPEED_MEAN$, 3.0) unif = ns3.RandomVariable(“Uniform”, 0, $ANGLE_BOUND$) node.setSpeed(norm) node.setAngle(unif) norm = ns3.RandomVariable(“Normal”, $SPEED_MEAN$, 3.0) node.setSpeed(norm) node.setAngle(ns3.Constant(270)) // A bunch of ns-3 code after… 69

70. Creating an ns-3 script template. Using the block element in NSTL, one can specify interchangeable code blocks to swap in an ns-3 script. The two different scripts are boxed in blue. // A bunch of ns-3 code before… norm = ns3.RandomVariable(“Normal”, 5.0, 3.0) unif = ns3.RandomVariable(“Uniform”, 0, 30) node.setSpeed(norm) node.setAngle(unif) // A bunch of ns-3 code after… // A bunch of ns-3 code before… norm = ns3.RandomVariable(“Normal”, 5.0, 3.0) node.setSpeed(norm) node.setAngle(ns3.Constant(270)) // A bunch of ns-3 code after… SPEED_MEAN = 5.0 ANGLE_BOUND = 30 Design Point 70

71. Concluding Thoughts 71

72. Concluding Thoughts NEDL and NSTL are markup languages; supporting modules can evolve. NEDL and NSTL not necessarily tied to ns-3. Working code generation a great benefit. 72

73. Future Work 73

74. Future Work GUIs for constructing NEDL documents. Repository of NSTL files created by experts. 74

75. Thank you! 75

76. 76

77. Creating an ns-3 script template. Using the block element in NSTL, one can specify interchangeable code blocks to swap in an ns-3 script. Since two block elements are used, two scripts will be generated. // BLOCK 1.1 // BLOCK 1.2 // BLOCKS 2.0, 2.1, 2.2, etc. 77