1. Automation of Testing in the Distributed Common Ground System (Army)
Michael Eng
Mentor: Mark Pumphrey, Greg Cordero
Northrop Grumman Mission Systems

2. Outline Purpose DCGS-A overview JUnit Automation of testing
Problems encountered
Results/conclusion

3. Purpose Explore methods of software testing and testing automation
Help to test and debug the DCGS-A project
Improve software quality

4. DCGS-A Distributed Common Ground System- Army Modular & decentralized
Integrate multiple types/sources of intelligence
Java, JavaBeans, Oracle
Similar efforts taking place in other branches of the military

5. Why Test? System crashes in field System is hacked
Loss of communication between commander and soldiers
No one knows where the enemy is
Intelligence database cannot be updated
System is hacked
Self-explanatory

6. JUnit Unit testing framework for Java
Test classes correspond to main classes
E.g., MyClass.java & MyClassTest.java
Run  status bar- green or red
Used with Eclipse to test DCGS-A
Perfect for testing individual instances of objects, methods, etc.

7. Methodology/Procedure
Each method with logic is tested
Every constructor is tested
Unit testing- method input arguments, conditional statements tested
Edge testing
Dummy objects
Hardcoded characteristics- force all possibilities
“logic…that is, not a simple get or set method”
THIS INCLUDES GARBAGE DATA AND RANDOMLY GENERATED DATA- FUZZ TESTING

8. Procedure continued JUnit4TestRunner, Logger JUnit formatting
JUnit @Test
Assertions
E.g., assertEquals(array[0], “Proper answer”);
assertNull([something that returns null]);
Expected
void test (expected =NullPointerException.class)
Exception testing
Javadoc notation is placed before the method to signify to the TestRunner that the method is a test method
Before is used to reset variables before each test
After usually is used to free up memory and resources or reset variables after each test

9. Example Test Program class myClassTest class myClass{ void testParse()
myClass.parse(null);
void testParse2()
myClass.parse(“”);
void testParse3()
myClass.parse(“ABC”);
void testParse4()
myClass.parse(“3##”);
void testParse5()
myClass.parse(“53”);
void testParse6()
myClass.parse(
new Object().toString());
void testParse7()
myClass.parse(new Integer(54));
void testParse8()
myClass.parse(new String[2]);
And so on…
class myClass{
static char parse (String s){
…logic to parse out the first number character in a String (e.g. ‘3’ in “ABC3WW5”)…
return that number as a char or null if not found }
Say we have a class MyClass with method parse that gets the first number character in an input String and returns it or null if there’s no numbers in the string. On the right we have a JUnit test class. So, we test the method by sending null. Then we send an empty String and expect a null output, or maybe an Exception if the method doesn’t check for that sort of thing (which we would then fix in the main class). Then we send a String that’s all letters and expect it to return null. Then a string that actually has a number. Then an all-number string. Then an Object forced into a String. Then an Integer object. Then a String array instead of an object. We could even go so far as to send a string one character too large (32767 is the limit) to see how gracefully the program handles overflows. And so on.

10. Procedure continued One method can have many, many cases
Vast majority of possibilities must be tested
Errors, if-else ladders, arguments, etc.
“Hardened” classes important
System must be protected
Reflection, buffer overflow attacks
“One method can have many, many cases”…and most, if not all need to be covered in testing
“System must be protected…tactics like reflection and data overflow attacks have to be handled- for example reflection can be stopped by making critical values private and final, and many overflow attacks can be stopped with bounds checking”

11. Automation Tests are written, now what?
JUnit AllTests- template for mass runs
Project has over 700 test methods
“Each subdirectory has an AllTests file that basically loads either the AllTests files below it, or loads each test class into it and runs the tests. It actually only takes a little under 10 seconds to run the whole system’s tests. So that saves a LOT of time.”

12. Problems Encountered External frameworks Test dependency hell
Undocumented or still being integrated
Test dependency hell
E.g., testing a class nested in another that depends on unavailable/unfinished properties file
Mock objects/files
Not really a problem, but constant updates to data classes still in development/pre-debugging
Constant updates of old tests

13. Results/Conclusion Created over 20 test classes
Modified over 10 existing test classes and main classes (bug fixes, etc.)
Still working on getting 100% function, path, and condition coverage for classes that need testing
Modifying existing tests to reflect updates in development