1. Automation of Software Testing for the Distributed Common Ground System – Army (DCGS-A)
Michael Eng
Mentors: Mark Pumphrey, Greg Cordero
Northrop Grumman Mission Systems

2. Outline Purpose DCGS-A overview JUnit Automation of Testing
Problems Encountered
Results/Conclusion

3. Explore methods of software testing and testing automation
Purpose
Explore methods of software testing and testing automation
Help to test and debug the DCGS-A project
Improve software quality
Meet functional and contractual requirements
Ensure functionality in deployment

4. What is DCGS-A? Distributed Common Ground System- Army
Signals Intelligence branch
SIGINT
Modular & decentralized
Integrate multiple types/sources of intelligence
Infrastructure of Java, JavaBeans, Oracle
Similar efforts taking place in other branches of the military

5. CPP Shelter on Armored M1152 –Assured Mobility for BCT, Div, and Above
Common Shelter in Battle Command

6. Why Test Software?
Software developed for the US Military must be, to the absolute extent possible, free of error because lives and mission are at stake.
System crashes in field
Loss of communication between commander and soldiers
No one knows where the enemy is
Intelligence database cannot be updated
System is hacked
Self-explanatory

7. JUnit Unit testing framework for Java
Part of xUnit family of frameworks
Test classes correspond to main classes
E.g., MyClass.java & MyClassTest.java
Test and main directories parallel each other
Run  status bar- green or red
Used with Eclipse to test DCGS-A
Perfect for testing individual instances of objects, methods, etc.

8. Methodology/Procedure
Each method with logic is tested
Every constructor is tested
Unit testing- method input arguments, conditional statements tested
Edge testing
Dummy objects
Hardcoded characteristics- force all possibilities
JUnit4TestRunner, Logger
JUnit formatting
JUnit @Test
Assertions
E.g., assertEquals(array[0], “Proper answer”);
assertNull([something that returns null]);
Expected
void test (expected =NullPointerException.class)
Exception testing
“logic…that is, not a simple get or set method”
THIS INCLUDES GARBAGE DATA AND RANDOMLY GENERATED DATA- FUZZ TESTING

9. Procedure Continued One method can have many, many cases
Vast majority of possibilities must be tested
Errors, if-else ladders, arguments, etc.
“Hardened” classes important
System must be protected
Reflection, buffer overflow attacks
Make sure values are private/final
Bounds checking for overflow attacks
“One method can have many, many cases”…and most, if not all need to be covered in testing
“System must be protected…tactics like reflection and data overflow attacks have to be handled- for example reflection can be stopped by making critical values private and final, and many overflow attacks can be stopped with bounds checking”

10. Example Test Program class myClass{ class myClassTest
static char parse (String s){
…logic to parse out the first number character in a String (e.g. ‘3’ in “ABC3WW5”)…
return that number as a char or null if not found }
class myClassTest
void testParse()
myClass.parse(null);
void testParse2()
myClass.parse(“”);
void testParse3()
myClass.parse(“ABC”);
void testParse4()
myClass.parse(“3##”);
void testParse5()
myClass.parse(“53”);
void testParse6()
myClass.parse(
new Object().toString());
void testParse7()
myClass.parse(new Integer(54));
void testParse8()
myClass.parse(new String[2]);
And so on…
Say we have a class MyClass with method parse that gets the first number character in an input String and returns it or null if there’s no numbers in the string. On the right we have a JUnit test class. So, we test the method by sending null. Then we send an empty String and expect a null output, or maybe an Exception if the method doesn’t check for that sort of thing (which we would then fix in the main class). Then we send a String that’s all letters and expect it to return null. Then a string that actually has a number. Then an all-number string. Then an Object forced into a String. Then an Integer object. Then a String array instead of an object. We could even go so far as to send a string one character too large (32767 is the limit) to see how gracefully the program handles overflows. And so on.

11. Automation Tests are written, now what?
JUnit AllTests- template for mass runs
Project has over 700 test methods
Before automation- run each test class manually
Hours
After automation- run one root test class
10 seconds
“Each subdirectory has an AllTests file that basically loads either the AllTests files below it, or loads each test class into it and runs the tests. It actually only takes a little under 10 seconds to run the whole system’s tests. So that saves a LOT of time.”

12. Problems Encountered External frameworks Test dependency hell
Undocumented or still being integrated
Test dependency hell
E.g., testing a class nested in another that depends on unavailable/unfinished properties file
Not really a problem, but constant updates to data classes still in development/pre-debugging
Constant updates of old tests

13. Results/Conclusion Created over 20 test classes
Modified over 10 existing test classes (bug/build fixes, etc.)
Found and fixed 10 bugs using test classes
Wrong classifications
Incorrect date calculations
Incorrect order of if-else ladders
Still working on getting 100% function, path, and condition coverage for classes that need testing
Modifying existing tests to reflect updates in development

14. Questions?