MEWT 2 13 th September 2014

Regression Testing: The Bane of the Tester Paul Berry (written on the train)

Initial thoughts When I first read what the theme of this year’s workshop was I thought “Well, that's a bit obvious. Surely everyone knows what Regression Testing is?” It turns out there’s no such thing as a silly question, especially when it comes to testing. So let me just set out what I understand regression testing to be, because I know already (thanks, Twitter) that there’s a difference of opinion in the room as to what that might be. Again, I’m not going off what the “dictionary definition” says. I’ve deliberately not looked it up. I must present what I understand it to be, not what I’d like it to be, and certainly not what some other source says it is.

Regression Testing is... Ensuring what has already been tested, and found to be good, remains so as the software continues to be developed. Ideally you would re-run all your previous tests every time there is a change.

What’s the problem? Why is it the “bane of the tester”?

What’s the problem? Why is it the “bane of the tester”? There is rarely (if ever) enough resource to cover a full regression test every time it is needed.

Reducing the load There are some common tactics to reduce the workload…

Reducing the load There are some common tactics to reduce the workload… 1.Don’t retest everything.

Reducing the load There are some common tactics to reduce the workload… 1.Don’t retest everything. Wait until there are enough changes built up to form a release, then test that. Reduces effort with minimal risk.

Reducing the load There are some common tactics to reduce the workload… 1.Don’t retest everything. Wait until there are enough changes built up to form a release, then test that. Reduces effort with minimal risk. Only test significant changes. How do you determine “significant”?

Reducing the load There are some common tactics to reduce the workload… 1.Don’t retest everything. Wait until there are enough changes built up to form a release, then test that. Reduces effort with minimal risk. Only test significant changes. How do you determine “significant”? Only test those areas likely to have been affected. Again, sensible, but how is this determined?

Reducing the load There are some common tactics to reduce the workload… 2.Don’t do it all yourself.

Reducing the load There are some common tactics to reduce the workload… 2.Don’t do it all yourself. Spread the testing effort out across the team. Rerunning tests manually is dull. Rotate the tests. Share the pain.

Reducing the load There are some common tactics to reduce the workload… 2.Don’t do it all yourself. Spread the testing effort out across the team. Rerunning tests manually is dull. Rotate the tests. Share the pain. Outsource the testing. Rerunning tests manually is dull. Let factory testers do it.

Reducing the load There are some common tactics to reduce the workload… 2.Don’t do it all yourself. Spread the testing effort out across the team. Rerunning tests manually is dull. Rotate the tests. Share the pain. Outsource the testing. Rerunning tests manually is dull. Let factory testers do it. Automate the testing. Let the computer do the work. Now you have two problems.

Maintain it Also remember to maintain the Regression Tests. Whether manual, automated, or a mix.

Coverage Regression Tests are…

Coverage Regression Tests are… All the tests you have run to check against requirements.

Coverage Regression Tests are… All the tests you have run to check against requirements. All the tests you have run to check the fixes are correct.

Coverage Regression Tests are… All the tests you have run to check against requirements. All the tests you have run to check the fixes are correct. That’s a lot of tests.

Regression Testing vs Sapient Testing Regression Testing is the antithesis of Sapient Testing.

Regression Testing vs Sapient Testing Regression Testing is the antithesis of Sapient Testing. Use sapient testing to explore the way ahead. Use regression testing to shore up the road behind.

Q&A Talk!