System Testing http://flic.kr/p/edctkJ Antares Rocket Test Launch (201304210013HQ) http://flic.kr/p/edctkJ System Testing

SWEBOK Knowledge Areas Software Requirements Software Design Software Construction Software Testing Software Maintenance Software Configuration Management Software Engineering Management Software Engineering Process Software Engineering Models and Methods Software Quality Software Engineering Professional Practice Software Engineering Economics Computing Foundations Mathematical Foundations Engineering Foundations Today’s topic

So you’re doing test-driven development – is it enough?

From the news! http://www.npr.org/blogs/alltechconsidered/2013/10/23/240242572/its-easy-to-blame-the-canadians-for-healthcare-gov-glitches

So you’re doing test-driven development – is it enough? Probably not. For one, TDD emphasizes unit tests, but what happens when you stick the units together?

System Testing Hook everything together Treat system like a black box Test it “end to end” Treat system like a black box Use real-world scenarios Focus on functionality What the customer asked for!

Developers typically do unit testing (as in TDD) Who should do system testing? The developers?

Developer Testing Developers know too much Can’t put themselves in the users shoes Especially never system test your own code It’s your baby

Tester Testing Dedicated testers bring fresh perspective More motivated to find bugs

System Testing + Iterative Development Two parallel iterations Bugs found by testers logged, scheduled in future iterations Moving target makes testing a challenge Communication key!

Principle: The key to most problems you’ll run into in software development is COMMUNICATION. When in doubt, TALK to your team, other teams, and your customer.

Other possible ways to coordinate testing Test iteration at the end Alternate dev/testing iterations What are pros/cons of these?

Tips for Effective Testing Document tests Do tests the same each time Establish success criteria When can system go live? Automate where possible Involve the customer Users always find problems the best testers miss

Now for a bonus verification technique that is not testing

Code Reviews Code walkthrough: Developer leads a review team through code Informal, focus on code Code inspection: Review team checks against a list of concerns Team members prepare offline in many cases Team moderator usually leads http://flic.kr/p/75HG5c

More on Code Reviews Some experiments show removal of 67-85% of defects via inspections Some consider XP’s pair programming as a kind of “code review” process, but it’s not quite the same Why? Can review/walkthrough requirements and design documents, not just code!