xUnit Test Patterns (Some) xUnit Test Patterns (in practice) by Adam Czepil.

Slides:



Advertisements
Similar presentations
Automating Software Module Testing for FAA Certification Usha Santhanam The Boeing Company.
Advertisements

Unit Testing Australian Development Centre Brisbane, Australia.
Designing Reusable Frameworks for Test Automation
SPL/2010 Test-Driven Development (TDD) 1. SPL/
Test process essentials Riitta Viitamäki,
Lectures on File Management
Javascript Code Quality Check Tools Javascript Code Quality Check Tools JavaScript was originally intended to do small tasks in webpages, but now JavaScript.
Software Development Languages and Environments. Programming languages High level languages are problem orientated contain many English words are easier.
Test-Driven Development and Refactoring CPSC 315 – Programming Studio.
Test-Driven Development. Why Testing is Important? “If you don’t have tests, how do you know your code is doing the thing right and doing the right thing?”
Taking a Waterfall Project Agile REF: Paul Geberth GCSS-J Project Manager Establishment of an Agile Project.
Developer Testing and Debugging. Resources Code Complete by Steve McConnell Code Complete by Steve McConnell Safari Books Online Safari Books Online Google.
How to Optimize Your Existing Regression Testing Arthur Hicken May 2012.
Debugging CPSC 315 – Programming Studio Fall 2008.
CS350/550 Software Engineering Lecture 1. Class Work The main part of the class is a practical software engineering project, in teams of 3-5 people There.
1 Today More on random testing + symbolic constraint solving (“concolic” testing) Using summaries to explore fewer paths (SMART) While preserving level.
Introduction to Software Engineering CS-300 Fall 2005 Supreeth Venkataraman.
Slide 1 of 9 Presenting 24x7 Scheduler The art of computer automation Press PageDown key or click to advance.
By for Test Driven Development: Industry practice and teaching tool Robert Vanderwall, Ph.D. 1 WISTPC-15.
Upstream Prerequisites
CSE 486/586 CSE 486/586 Distributed Systems PA Best Practices Steve Ko Computer Sciences and Engineering University at Buffalo.
Lecture 6 Software Testing and jUnit CS140 Dick Steflik.
Introduction to Unit Testing Jun-Ru Chang 2012/05/03.
Software Testing CS 121 “Ordering Chaos” “Mike” Michael A. Erlinger
1. Topics to be discussed Introduction Objectives Testing Life Cycle Verification Vs Validation Testing Methodology Testing Levels 2.
Designing For Testability. Incorporate design features that facilitate testing Include features to: –Support test automation at all levels (unit, integration,
Chapter 8 – Software Testing Lecture 1 1Chapter 8 Software testing The bearing of a child takes nine months, no matter how many women are assigned. Many.
Winrunner Usage - Best Practices S.A.Christopher.
T-unit: Tcl Unit Test Package Automated Unit Test Package For Tcl Procedures Final Presentation Joseph Boyle Loyola Marymount University.
Dr. Tom WayCSC Testing and Test-Driven Development CSC 4700 Software Engineering Based on Sommerville slides.
1 Performance Optimization In QTP Execution Over Video Automation Testing Speaker : Krishnesh Sasiyuthaman Nair Date : 10/05/2012.
From Quality Control to Quality Assurance…and Beyond Alan Page Microsoft.
DEBUGGING. BUG A software bug is an error, flaw, failure, or fault in a computer program or system that causes it to produce an incorrect or unexpected.
(1) Unit Testing and Test Planning CS2110: SW Development Methods These slides design for use in lab. They supplement more complete slides used in lecture.
CS5103 Software Engineering Lecture 02 More on Software Process Models.
Test-Driven Development Eduard Miric ă. The problem.
12/14/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 1 Automated Testing Environment Concepts.
HOW AND WHY TO LOVE CUCUMBER By Dana Scheider. Is This Your Programming Experience?
Software Quality Assurance and Testing Fazal Rehman Shamil.
Version Control and SVN ECE 297. Why Do We Need Version Control?
The single most important skill for a computer programmer is problem solving Problem solving means the ability to formulate problems, think creatively.
Integrating the Code during the Development Alexander Vakrilov Telerik Corporation
Unit Testing with FlexUnit
Beginning Software Craftsmanship Brendan Enrick Steve Smith
Test Automation Steffen Goerlitz Barry Lange Mitchell Meerman Harry Schultz Trevor Spees.
1 Punishment Through Continuous Delivery If it hurts, do it more often…
Beginning Software Craftsmanship Brendan Enrick Steve Smith
Automated Testing with PHPUnit. How do you know your code works?
Software Engineering Lecture 11 Software Testing Presenter: Josef Hallberg 1.
The Troubleshooting Process. Hardware Maintenance Make sure that the hardware is operating properly.  Check the condition of parts.  Repair or replace.
Automated Software Testing
Unit Testing - solid fundamentals
Designing For Testability
Test Driven Development
Chapter 8 – Software Testing
Systematic Manual Testing
Unit testing C# classes
Some Important Techniques For Regression Testing That You Must Know.
Automated Testing Environment
CSCE 315 – Programming Studio, Fall 2017 Tanzir Ahmed
Testing and Test-Driven Development CSC 4700 Software Engineering
CS240: Advanced Programming Concepts
Regression testing Tor Stållhane.
An Introduction to Debugging
Applying Use Cases (Chapters 25,26)
Case Study 1 By : Shweta Agarwal Nikhil Walecha Amit Goyal
Applying Use Cases (Chapters 25,26)
Test Cases, Test Suites and Test Case management systems
Test-Driven Development
Serverless Computing: Promises & Pitfalls
Presentation transcript:

xUnit Test Patterns

(Some) xUnit Test Patterns (in practice) by Adam Czepil

Who needs these? What kind of problems xUnit test patterns may help to solve? Who will benefit the most?

Goals of Test Automation Code Smells Principles of Test Automation Behavior Smells Projects Smells Test Patterns

Is it a fish?

Or is it a mammal?

Revenons a nos moutons!

What is Defect Localization and why developers in good projects (almost) dont use debugger? Debugging is an art

The problem

We run the test (possibly manual test)

The problem We run the test (possibly manual test) The test fails

The problem We run the test (possibly manual test) The test fails We have to figure out why it fails?

The problem We run the test (possibly manual test) The test fails We have to figure out why it fails? The problem is that this can be very time consuming...

Fortunatelly we have extra-skilled developers who can find the problem very quickly ;-)

What we expect from very good tests is that

if the tests fail they provide information

What we expect from very good tests is that if the tests fail they provide information detailed enough so we can find the source of the failure quickly.

Defect Localization ability to very quickly find the actual cause of test failure

Defect Localization is

one of the major goals of test automation!

Defect Localization is

one of the major advantages of unit tests over customer (functional) tests!

What to do to have good Defect Localization?

Frequent Debugging

When we deal with Frequent Debugging?

When manual debugging is required to determine the cause of most test failures.

When we deal with Frequent Debugging? When manual debugging is required to determine the cause of most test failures. This is also true when instead of debugging we use extensive logging during testing.

Frequent Debugging...

reduces productivity!

Frequent Debugging... reduces productivity! and makes development schedules much less predictable!

Frequent Debugging... reduces productivity! and makes development schedules much less predictable! because a single manual debugging session could extend the time required to develop the software by half a day or more.

What are the causes of Frequent Debugging?

Frequent Debugging is caused by

What are the causes of Frequent Debugging? Frequent Debugging is caused by missing tests or

What are the causes of Frequent Debugging? Frequent Debugging is caused by missing tests or lack of Defect Localization in our suite of automated tests.

We should be able to quickly determine the root cause of test failure based on

We should be able to quickly determine the root cause of test failure based on Assertion Message

We should be able to quickly determine the root cause of test failure based on Assertion Message or through the pattern of test failures

We should be able to quickly determine the root cause of test failure based on Assertion Message or through the pattern of test failures and knowledge about changes being made.

What are the main causes for no Defect Localization?

not enough detailed unit tests compared to functional tests,

What are the main causes for no Defect Localization? not enough detailed unit tests compared to functional tests, missing component tests (for a cluster of classes) that would point out an integration error between classes.

What are the main causes for no Defect Localization? not enough detailed unit tests compared to functional tests, missing component tests (for a cluster of classes) that would point out an integration error between classes, Infrequently Run Tests, because it causes that it is difficult to point out the change that caused the tests to fail.

Very good solution for Frequent Debugging problem is...

TDD – Test Driven Development

Very good solution for Frequent Debugging problem is... TDD – Test Driven Development (or at least Test First)

What I want you to remember?

Defect Localization is about finding quickly bug causing the tests to fail.

What I want you to remember? Defect Localization is about finding quickly bug causing the tests to fail. Unit tests are usually much better in Defect Localization than customer tests.

What I want you to remember? Defect Localization is about finding quickly bug causing the tests to fail. Unit tests are usually much better in Defect Localization than customer tests. Frequent Debugging usually means that project is either missing tests or has poor Defect Localization.

What I want you to remember? Defect Localization is about finding quickly bug causing the tests to fail. Unit tests are usually much better in Defect Localization than customer tests. Frequent Debugging usually means that project is either missing tests or has poor Defect Localization. Use TDD as often as possible

WARNING! Too much debugging may broke your eyes! ;-)

Questions?

How NOT to automate manual tests? What are some differences between manual and automated test-case? How to create automated test comparing to manual test?

Manual vs. Automated Tests

Lets compare some aspects...

Manual vs. Automated Tests Lets compare some aspects... -time

Manual vs. Automated Tests Lets compare some aspects... -time, -fragility.

Time Executing manual tests requires A LOT OF time from HUMAN

Time Executing automated tests requires almost NO time from HUMAN

Since HUMAN time is precious and COMPUTER time is cheap we should care much about the first and dont have to worry too much about the latter.

Conclusions Opposite to manual tests we can create unlimited number of regression test cases.

Fragility When the test fails/stops in some non-final point HUMAN usually finds a way (work around) to continue its execution.

Fragility When the test fails in some non-final point test execution STOPS.

HUMAN, although much slower than COMPUTER, is smarter test executor, especially when the test fails before all test conditions have been checked.

Conclusions Opposite to manual tests we have to care about assertions order and find a way of handling not executed assertions.

Verify One Condition per Test

Verify One Condition per Test principle is not the same as One Assertion Per Test principle

Lets see an example... exampleexample refactored

When (not) to use setup and teardown in xUnit?

Verify One Condition per Test helps to give us good Defect Localization to pinpoint the source of the failure.

Another reason we Verify One Condition per Test is to Minimize Test Overlap

Another reason we Verify One Condition per Test is to Minimize Test Overlap, so that we have fewer tests to modify if we later modify the behavior of the SUT.

So? How NOT to automate manual tests?

Dont write long, complex test scenarios

So? How NOT to automate manual tests? Dont write long, complex test scenarios, instead Verify One Condition per Test

So? How NOT to automate manual tests? Dont write long, complex test scenarios, instead Verify One Condition per Test, but not necesserily one assertion per test.

Questions?

Hey! Test code is code too! Is test code different than regular code? How to organize test cases into classes/files/directories? What about test utils/helpers? Where to keep them?

Questions?

There is a lot of patterns described in the book. Here are maps of them...

The Maps

Why its worthwile to study xUnit patterns?

Questions?

Thank you! Go there: Really! You dont have to read everything there...