The DDS Benchmarking Environment James Edmondson Vanderbilt University Nashville, TN

2 Topics What is the DBE? What technologies does the DBE use? How does the DBE setup the environment? What problems had to be dealt with? What features will be added? Demonstration Conclusion and Wrap Up

3 What is the DBE? DDS Benchmarking Environment Sets up the testing environment and framework Configures and launches implementation specific services Launches Publishers and Subscribers Coordinates interaction, logging info, and stats collection Allows for easily scaling the number of participants Sets up the statistics for collection by Bagatel Throughput Publishers Networking Subscribers

4 What is the DBE? May be used to test other middleware or technologies Focus is on documented, reusable resources Throughput Publishers Networking Subscribers VersatilityDependability

5 What does the DBE do?

6

7

8

9

10 What does the DBE do? Start the Subscribers

11 What does the DBE do?

12 What does the DBE do?

13 What does the DBE do?

14 What does the DBE do?

15 What does the DBE do? Start the Publisher

16 What does the DBE do?

17 What does the DBE do?

18 What technologies does the DBE use? Scripts are written in Perl and use PerlAce module Remote execution is accomplished via SSH Logs, stats and test info are stored on a NFS partition All tests are currently ran on Fedora Core Linux machines Configured to test NDDS, TAO DDS, and Splice Synchronization between tests is maintained by using lockfiles DBE LinuxSSHNFSPerllockfiles

19 How does the DBE setup the environment? Most path information is updated using the ~/.ssh/environment file Some custom environment variables are set by DBE scripts on individual clients Individual Test IDs are assigned to each test Directories are created on the NFS partition for stats and logs DBE

20 How does the DBE setup the environment? Most path information is updated using the ~/.ssh/environment file Some custom environment variables are set by DBE scripts on individual clients Individual Test IDs are assigned to each test Directories are created on the NFS partition for stats and logs DBE

21 How does the DBE setup the environment? Most path information is updated using the ~/.ssh/environment file Some custom environment variables are set by DBE scripts on individual clients Individual Test IDs are assigned to each test Directories are created on the NFS partition for stats and logs DBE Change $ENV

22 How does the DBE setup the environment? Most path information is updated using the ~/.ssh/environment file Some custom environment variables are set by DBE scripts on individual clients Individual Test IDs are assigned to each test Directories are created on the NFS partition for stats and logs DBE Change $ENV

23 How does the DBE setup the environment? Most path information is updated using the ~/.ssh/environment file Some custom environment variables are set by DBE scripts on individual clients Individual Test IDs are assigned to each test Directories are created on the NFS partition for stats and logs DBE Change $ENVGenerate ID

24 How does the DBE setup the environment? Most path information is updated using the ~/.ssh/environment file Some custom environment variables are set by DBE scripts on individual clients Individual Test IDs are assigned to each test Directories are created on the NFS partition for stats and logs DBE Change $ENVGenerate ID

25 How does the DBE setup the environment? Most path information is updated using the ~/.ssh/environment file Some custom environment variables are set by DBE scripts on individual clients Individual Test IDs are assigned to each test Directories are created on the NFS partition for stats and logs DBE Change $ENVGenerate IDCreate Directories

26 How does the DBE setup the environment? Publishers and subscribers are notified of test settings –Result file locations –QoS settings on a per subscriber basis –Network settings –Test specific settings (number of messages to send, etc.) DBE ReposPublishersSubscribers Pass Test Info

27 What problems had to be dealt with? Lack of a terminal with remote command execution with SSH TCP and UDP buffer sizes were too small Synchronization amongst publishers and subscribers Subscriber Publisher Wait Publisher Buffer Publisher Buffer Next Test

28 What problems had to be dealt with? Flock() doesn’t work as expected with stateless NFS Shared libraries are referenced locally Block buffering of stdout flock(“myfile”)touch(“myfile.lock”) stdout stderr file immediate block buffering

29 What problems had to be dealt with? More problems on request ;)

30 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE

31 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE Create Publisher

32 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE Create Publisher

33 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE Publisher Create Publisher

34 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE Publisher

35 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE Publisher Create Subscriber

36 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE Publisher Create Subscriber

37 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE PublisherSubscriber Create Subscriber

38 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE PublisherSubscriber

39 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE PublisherSubscriber Create Subscriber

40 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE PublisherSubscriber Create Subscriber

41 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE PublisherSubscriber Create Subscriber

42 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE PublisherSubscriber

43 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE PublisherSubscriber Kill Subscriber 1

44 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE PublisherSubscriber Kill Subscriber 1

45 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE PublisherSubscriber Kill Subscriber 1

46 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE PublisherSubscriber Kill Subscriber 1

47 What features will be added? New tests will require new features One such feature is per node scripting of subscribers and publishers Feature will allow dynamic starting and stopping of participants DBE PublisherSubscriber

48 What other additions are planned? The ability to launch logging services –Memory and CPU usage will need to be recorded The ability to test other middleware and applications

49 Demonstration Primer messages = 100 Stats messages = 1000 Test type = throughput Implementation = NDDS

50 Conclusion and Wrap Up The DBE is an excellent suite of tools for automating DDS testing on a networked environment New test types will be added to gain further insight into the DDS implementations Many new features will also be added to better reveal problems and benefits of the various DDS products We would like to thank our AFRL and Vanderbilt Sponsors for their continued support