Test Driven Development: An Emerging Solution for Software Development

Introduction Traditional Approach Traditional Approach Test last Test last Problems with Traditional Problems with Traditional Errors in production Errors in production Programmer moves onto other projects Programmer moves onto other projects Test and code written by different programmers Test and code written by different programmers Tests based on outdated information Tests based on outdated information Infrequent testing Infrequent testing Fixes that create other problems Fixes that create other problems

Aspects of TDD Features Features High level user requirements High level user requirements User story User story Customer Tests Customer Tests Customer identified acceptance tests Customer identified acceptance tests Developer Tests Developer Tests Tests developed during software construction Tests developed during software construction

Methodology Test first – Code last Test first – Code last You may not write production code unless you’ve first written a failing unit test You may not write production code unless you’ve first written a failing unit test Test more – Code more Test more – Code more You may not write more of a unit test than is sufficient to fail You may not write more of a unit test than is sufficient to fail Test again – Code again Test again – Code again You may not write more production code than is sufficient to make the failing unit test pass You may not write more production code than is sufficient to make the failing unit test pass

Advantages of TDD Regression testing Regression testing Reduced software defect rates Reduced software defect rates User requirements more easily understood User requirements more easily understood Reduced interface misunderstandings Reduced interface misunderstandings Test-lag avoided Test-lag avoided

Disadvantages of TDD Programmers like to code, not to test Programmers like to code, not to test Test writing is time consuming Test writing is time consuming Test completeness is difficult to judge Test completeness is difficult to judge TDD may not always work TDD may not always work

Test-Driven Learning (TDL) “Teach by example” “Teach by example” Present examples with automated tests Present examples with automated tests Start with tests Start with tests Teach testing for free Teach testing for free Teach automated testing frameworks simply Teach automated testing frameworks simply Encourage the use of test-driven development Encourage the use of test-driven development Improve the student comprehension and programming abilities Improve the student comprehension and programming abilities Improve software quality both in terms of design and defect density Improve software quality both in terms of design and defect density

Test-Driven Learning (TDL) #include #include using namespace std; int main() { int age; cout << “What is your age in years? << endl; << endl; cin >> age; cout << “You are at least “ << age * 12 << age * 12 << “ months old!” << endl; << “ months old!” << endl;} #include #include int main() { int age = 18; int age = 18; int ageInMonths; int ageInMonths; ageInMonths = age * 12; ageInMonths = age * 12; assert(ageInMonths == 216); assert(ageInMonths == 216);} Traditional:TDL:

Test-Driven Learning (TDL) #include #include using namespace std; int main() {assert(sum(3,7)==25); cout << “No errors encountered” << endl; } // This function sums the integers // from min to max inclusive // Pre: min < max // Post: return-value = min + (min +1) + … // + (max-1) + max int sum(int min, int max) { int sum = 0; int sum = 0; for(int i=min; i<=max; i++) for(int i=min; i<=max; i++) { sum += I sum += I } return sum; return sum;} assert(sum(-2,2) == 0); assert(sum(-4,2) == -9);

Implementation in Assignments Marmoset Marmoset Runs student code against teacher tests Runs student code against teacher tests Gives immediate feedback Gives immediate feedback Supports repeatable submitting Supports repeatable submitting Web-CAT Web-CAT Uses three scores: Test Validity, Test Completeness, Code Correctness Uses three scores: Test Validity, Test Completeness, Code Correctness Gives immediate feedback Gives immediate feedback Supports repeatable submitting Supports repeatable submitting

Study Findings Increase in code quality Increase in code quality Increase in productivity Increase in productivity Increase in programmer confidence Increase in programmer confidence Increase in program understanding Increase in program understanding

IBM’s Use of TDD Quality Using [TDD], we reduced our defect rate by about 50 percent compared to a similar system that was built using an ad-hoc unit testing approach. -IBM Low defect density errors per KLOC A Suite of Automated Tests is a reusable and extendable asset that will continue to improve quality over the lifetime of the software system. -IBM A nightly build that runs all tests should become the heartbeat of the system as well as a means to track progress of the development. -IBM

Additional Benefits Maintenance The fix for a JavaPOS defect generally does not break anything else and that fix [solves] the problem described in the defect description... As far as I have seen, testing other products of the same sort there is an important risk that a fix breaks something else or the fix does not solve the problem. –IBM Tester Tests provide documentation Tests define the functionality of the code Easily accessible for programmers Moving unit testing before implementation Results in improved testing Tests are more accurate and precise IBM noticed that prior to TDD “the unit test process was not disciplined and was done as an afterthought. More often than not, no unit tests were created” Provides immediate feedback for developers

Drawbacks and Ethics Drawbacks Increased time and effort up-front 50%+ of code is tests Microsoft found that TDD took up 15% their development time IBM, however, argues that “time previously spent on relatively unpredictable debugging was traded off for controlled test creation and execution.” Tests require upkeep and occasionally need to be refactored Ethics When quality is an ethical concern, whether or not to use TDD can be an ethical decision Critical systems Software engineers should approve software only if they have a well-founded belief that it is safe, meets specifications, passes appropriate tests, and does not diminish quality of life. -SE Code of Ethics, ACM