Unit Testing Fundamentals and NUnit Svetlin Nakov National Academy for Software Development academy.devbg.org Veselin Georgiev National Academy for Software Development academy.devbg.org

ContentsContents 1.Unit Testing Fundamentals Some factsSome facts Why unit tests?Why unit tests? 2.Unit Testing Patterns 3.NUnit Testing Framework AssertionsAssertions AttributesAttributes NUnit GUI ApplicationNUnit GUI Application Unit Testing Best PracticesUnit Testing Best Practices 1.Unit Testing Fundamentals Some factsSome facts Why unit tests?Why unit tests? 2.Unit Testing Patterns 3.NUnit Testing Framework AssertionsAssertions AttributesAttributes NUnit GUI ApplicationNUnit GUI Application Unit Testing Best PracticesUnit Testing Best Practices

DefinitionDefinition A unit test is a piece of code written by a developer that exercises a very small, specific area of functionality of the code being tested. “Program testing can be used to show the presence of bugs, but never to show their absence!” - Edsger Dijkstra, [1972]

Unit Test Example int Sum(int[] array) { sum = 0; for (int i=0; i<array.Length; i++) sum += array[i]; return sum; } void TestSum() { if (Sum(new int[]{1,2}) != 3) throw new TestFailedException("1+2 != 3"); if (Sum(new int[]{-2}) != -2) throw new TestFailedException("-2 != -2"); if (Sum(new int[]{}) != 0) throw new TestFailedException("0 != 0"); }

Some Facts Tests are specific pieces of codeTests are specific pieces of code Unit testing framework is neededUnit testing framework is needed NUnit, MbUnit, VS Unit Testing FrameworkNUnit, MbUnit, VS Unit Testing Framework Unit tests are written by developers for developersUnit tests are written by developers for developers Unit tests are released into the code repository along with the code they testUnit tests are released into the code repository along with the code they test

All classes should be testedAll classes should be tested All methods should be testedAll methods should be tested Trivial code may be omittedTrivial code may be omitted E.g. property getters and settersE.g. property getters and setters Ideally all unit tests should pass before check-inIdeally all unit tests should pass before check-in All classes should be testedAll classes should be tested All methods should be testedAll methods should be tested Trivial code may be omittedTrivial code may be omitted E.g. property getters and settersE.g. property getters and setters Ideally all unit tests should pass before check-inIdeally all unit tests should pass before check-in More Facts

Write code Write unit test Run and succeed Code and Test Approach Time flow

Pick а test Compile and Fail Write code to pass test Write enough code to compile Run test and fail Create a test list Test Driven Development Time flow Write test Remove duplication

Why Test Driven Development? Helps find design issues early and avoids reworkHelps find design issues early and avoids rework Writing code to satisfy a test is a focused activity – less chance of errorWriting code to satisfy a test is a focused activity – less chance of error Tests will be a more comprehensive than when written after codeTests will be a more comprehensive than when written after code Helps find design issues early and avoids reworkHelps find design issues early and avoids rework Writing code to satisfy a test is a focused activity – less chance of errorWriting code to satisfy a test is a focused activity – less chance of error Tests will be a more comprehensive than when written after codeTests will be a more comprehensive than when written after code

Unit tests dramatically decrease the number of defects in the codeUnit tests dramatically decrease the number of defects in the code Unit tests improve designUnit tests improve design Unit tests are good documentationUnit tests are good documentation Unit tests reduce the cost of changeUnit tests reduce the cost of change Unit tests allow refactoringUnit tests allow refactoring Unit tests decrease the defect-injection rate due to refactoring/changesUnit tests decrease the defect-injection rate due to refactoring/changes Unit tests dramatically decrease the number of defects in the codeUnit tests dramatically decrease the number of defects in the code Unit tests improve designUnit tests improve design Unit tests are good documentationUnit tests are good documentation Unit tests reduce the cost of changeUnit tests reduce the cost of change Unit tests allow refactoringUnit tests allow refactoring Unit tests decrease the defect-injection rate due to refactoring/changesUnit tests decrease the defect-injection rate due to refactoring/changes Why Unit Tests?

Unit Testing Patterns

Unit test patterns cover broad aspects of developmentUnit test patterns cover broad aspects of development May promote unit testing to become a more formal engineering disciplineMay promote unit testing to become a more formal engineering discipline Helps identify the kind of unit tests to write, and its usefulnessHelps identify the kind of unit tests to write, and its usefulness Allows developer to choose how detailed the unit tests need to beAllows developer to choose how detailed the unit tests need to be Unit test patterns cover broad aspects of developmentUnit test patterns cover broad aspects of development May promote unit testing to become a more formal engineering disciplineMay promote unit testing to become a more formal engineering discipline Helps identify the kind of unit tests to write, and its usefulnessHelps identify the kind of unit tests to write, and its usefulness Allows developer to choose how detailed the unit tests need to beAllows developer to choose how detailed the unit tests need to be

Unit Testing Patterns Categorization Pass/fail patternsPass/fail patterns Collection management patternsCollection management patterns Data driven patternsData driven patterns Performance patternsPerformance patterns Process patternsProcess patterns Simulation patternsSimulation patterns Multithreading patternsMultithreading patterns Pass/fail patternsPass/fail patterns Collection management patternsCollection management patterns Data driven patternsData driven patterns Performance patternsPerformance patterns Process patternsProcess patterns Simulation patternsSimulation patterns Multithreading patternsMultithreading patterns

Unit Testing Patterns Categorization (2) Stress test patternsStress test patterns Presentation layer patternsPresentation layer patterns Overview of all patterns can be found at:Overview of all patterns can be found at: ure/autp5.aspx ure/autp5.aspx Stress test patternsStress test patterns Presentation layer patternsPresentation layer patterns Overview of all patterns can be found at:Overview of all patterns can be found at: ure/autp5.aspx ure/autp5.aspx

Unit Testing Frameworks and NUnit

Unit Testing Frameworks JUnit – first Unit Testing FrameworkJUnit – first Unit Testing Framework Based on JavaBased on Java Unit Testing Frameworks have been developed for a broad range of computer languagesUnit Testing Frameworks have been developed for a broad range of computer languages NUnit – for C# and all.NET languagesNUnit – for C# and all.NET languages cppUnit, jsUnit, PhpUnit, PerlUnit,...cppUnit, jsUnit, PhpUnit, PerlUnit,... JUnit – first Unit Testing FrameworkJUnit – first Unit Testing Framework Based on JavaBased on Java Unit Testing Frameworks have been developed for a broad range of computer languagesUnit Testing Frameworks have been developed for a broad range of computer languages NUnit – for C# and all.NET languagesNUnit – for C# and all.NET languages cppUnit, jsUnit, PhpUnit, PerlUnit,...cppUnit, jsUnit, PhpUnit, PerlUnit,...

AssertionsAssertions Predicate is a true–false statementPredicate is a true–false statement Assertion is a predicate placed in a program codeAssertion is a predicate placed in a program code Indicates that the developer thinks that the predicate is always true at that placeIndicates that the developer thinks that the predicate is always true at that place If an assertion fails, the method call does not return and an error is reportedIf an assertion fails, the method call does not return and an error is reported Predicate is a true–false statementPredicate is a true–false statement Assertion is a predicate placed in a program codeAssertion is a predicate placed in a program code Indicates that the developer thinks that the predicate is always true at that placeIndicates that the developer thinks that the predicate is always true at that place If an assertion fails, the method call does not return and an error is reportedIf an assertion fails, the method call does not return and an error is reported

NUnit – Features Test code is annotated using custom attributesTest code is annotated using custom attributes Two execution interfacesTwo execution interfaces GUI: nunit-gui.exeGUI: nunit-gui.exe Console: nunit-console.exeConsole: nunit-console.exe NUnit provides:NUnit provides: Creating and running tests as NUnit Test ProjectsCreating and running tests as NUnit Test Projects Visual Studio supportVisual Studio support Test code is annotated using custom attributesTest code is annotated using custom attributes Two execution interfacesTwo execution interfaces GUI: nunit-gui.exeGUI: nunit-gui.exe Console: nunit-console.exeConsole: nunit-console.exe NUnit provides:NUnit provides: Creating and running tests as NUnit Test ProjectsCreating and running tests as NUnit Test Projects Visual Studio supportVisual Studio support

NUnit – Attribute Listing [TestFixture][TestFixture] [TestFixtureSetUp][TestFixtureSetUp] [TestFixtureTearDown][TestFixtureTearDown] [Test][Test] [SetUp][SetUp] [TearDown][TearDown] [ExpectedException(typeof(Exception ))][ExpectedException(typeof(Exception ))] [Ignore("message")][Ignore("message")] [TestFixture][TestFixture] [TestFixtureSetUp][TestFixtureSetUp] [TestFixtureTearDown][TestFixtureTearDown] [Test][Test] [SetUp][SetUp] [TearDown][TearDown] [ExpectedException(typeof(Exception ))][ExpectedException(typeof(Exception ))] [Ignore("message")][Ignore("message")]

NUnit – Attributes [TestFixture] – Marks a class that contains tests[TestFixture] – Marks a class that contains tests Should be a public classShould be a public class Must have a default constructorMust have a default constructor [Test] – Marks a method as a unit test[Test] – Marks a method as a unit test [TestFixtureSetUp] – Indicates a setup method that will be ran once before all other tests[TestFixtureSetUp] – Indicates a setup method that will be ran once before all other tests Called before the tests are startedCalled before the tests are started [TestFixture] – Marks a class that contains tests[TestFixture] – Marks a class that contains tests Should be a public classShould be a public class Must have a default constructorMust have a default constructor [Test] – Marks a method as a unit test[Test] – Marks a method as a unit test [TestFixtureSetUp] – Indicates a setup method that will be ran once before all other tests[TestFixtureSetUp] – Indicates a setup method that will be ran once before all other tests Called before the tests are startedCalled before the tests are started

NUnit – Attributes (2) [TestFixtureTearDown] – Indicates a tear down method that will be ran once after all other tests have run[TestFixtureTearDown] – Indicates a tear down method that will be ran once after all other tests have run Called after all the tests have finishedCalled after all the tests have finished [SetUp] – Indicates a setup method that should be ran before each of the tests[SetUp] – Indicates a setup method that should be ran before each of the tests [TearDown] – Indicates a tear down method that should be ran after each of the tests[TearDown] – Indicates a tear down method that should be ran after each of the tests [TestFixtureTearDown] – Indicates a tear down method that will be ran once after all other tests have run[TestFixtureTearDown] – Indicates a tear down method that will be ran once after all other tests have run Called after all the tests have finishedCalled after all the tests have finished [SetUp] – Indicates a setup method that should be ran before each of the tests[SetUp] – Indicates a setup method that should be ran before each of the tests [TearDown] – Indicates a tear down method that should be ran after each of the tests[TearDown] – Indicates a tear down method that should be ran after each of the tests

NUnit – Attributes (3) [ExpectedException(typeof(Except ion))][ExpectedException(typeof(Except ion))] When you expect an exception to be thrownWhen you expect an exception to be thrown Will only pass if exception type was thrownWill only pass if exception type was thrown [Ignore("Not ready")] – Indicates a test that is not ready, or should not be run[Ignore("Not ready")] – Indicates a test that is not ready, or should not be run [ExpectedException(typeof(Except ion))][ExpectedException(typeof(Except ion))] When you expect an exception to be thrownWhen you expect an exception to be thrown Will only pass if exception type was thrownWill only pass if exception type was thrown [Ignore("Not ready")] – Indicates a test that is not ready, or should not be run[Ignore("Not ready")] – Indicates a test that is not ready, or should not be run

NUnit – Attributes (4) Explicit tests (ignored unless run explicitly)Explicit tests (ignored unless run explicitly) [TestFixture, Explicit][TestFixture, Explicit] [Test, Explicit][Test, Explicit] Categories (used for grouping tests)Categories (used for grouping tests) [TestFixture, Category("LongRunning")][TestFixture, Category("LongRunning")] [Test, Category("Long")][Test, Category("Long")] Explicit tests (ignored unless run explicitly)Explicit tests (ignored unless run explicitly) [TestFixture, Explicit][TestFixture, Explicit] [Test, Explicit][Test, Explicit] Categories (used for grouping tests)Categories (used for grouping tests) [TestFixture, Category("LongRunning")][TestFixture, Category("LongRunning")] [Test, Category("Long")][Test, Category("Long")]

SetUp & TearDown Timeline Begin TestsBegin Tests Call TestFixtureSetUpCall TestFixtureSetUp Call SetUpCall SetUp Call ExampleTestOneCall ExampleTestOne Call TearDownCall TearDown Call SetUpCall SetUp Call ExampleTestTwoCall ExampleTestTwo Call TearDownCall TearDown Call TestFixtureTearDownCall TestFixtureTearDown End TestsEnd Tests Begin TestsBegin Tests Call TestFixtureSetUpCall TestFixtureSetUp Call SetUpCall SetUp Call ExampleTestOneCall ExampleTestOne Call TearDownCall TearDown Call SetUpCall SetUp Call ExampleTestTwoCall ExampleTestTwo Call TearDownCall TearDown Call TestFixtureTearDownCall TestFixtureTearDown End TestsEnd Tests

NUnit – Assertions Using NUnit.Framework.Assert classUsing NUnit.Framework.Assert class Assertions check condition and throw exception if condition is not satisfiedAssertions check condition and throw exception if condition is not satisfied Comparing valuesComparing values AreEqual(expected value, calculated value [,message]) – Compare two values for equalityAreEqual(expected value, calculated value [,message]) – Compare two values for equality Comparing objectsComparing objects IsNull(object [,message])IsNull(object [,message]) Using NUnit.Framework.Assert classUsing NUnit.Framework.Assert class Assertions check condition and throw exception if condition is not satisfiedAssertions check condition and throw exception if condition is not satisfied Comparing valuesComparing values AreEqual(expected value, calculated value [,message]) – Compare two values for equalityAreEqual(expected value, calculated value [,message]) – Compare two values for equality Comparing objectsComparing objects IsNull(object [,message])IsNull(object [,message])

NUnit – Assertions (2) IsNotNull(object [,message])IsNotNull(object [,message]) AreSame(expected object, current object [,message]) – Compares object referencesAreSame(expected object, current object [,message]) – Compares object references ConditionsConditions IsTrue(condition)IsTrue(condition) IsFalse(condition)IsFalse(condition) Forced test failForced test fail Fail()Fail() IsNotNull(object [,message])IsNotNull(object [,message]) AreSame(expected object, current object [,message]) – Compares object referencesAreSame(expected object, current object [,message]) – Compares object references ConditionsConditions IsTrue(condition)IsTrue(condition) IsFalse(condition)IsFalse(condition) Forced test failForced test fail Fail()Fail()

NUnit – Example public class Account { private float balance; private float balance; public void Deposit(float amount) public void Deposit(float amount) { balance += amount; balance += amount; } public void Withdraw(float amount) public void Withdraw(float amount) { balance -= amount; balance -= amount; } public void TransferFunds( public void TransferFunds( Account destination, float amount) Account destination, float amount) {... } {... } public float Balance public float Balance {... } {... }}

NUnit – Example (2) using NUnit.Framework; [TestFixture] public class AccountTest { [Test] [Test] public void TransferFunds() public void TransferFunds() { Account source = new Account(); Account source = new Account(); source.Deposit(200.00F); source.Deposit(200.00F); Account dest = new Account(); Account dest = new Account(); dest.Deposit(150.00F); dest.Deposit(150.00F); source.TransferFunds(dest, F); source.TransferFunds(dest, F); Assert.AreEqual(250.00F, dest.Balance); Assert.AreEqual(250.00F, dest.Balance); Assert.AreEqual(100.00F, source.Balance); Assert.AreEqual(100.00F, source.Balance); }}

NUnit – Screen Shots

TestDriven – A Helpful Tool For developers who want to work in the Visual Studio IDE there is the add-onFor developers who want to work in the Visual Studio IDE there is the add-on Tests can be ran with a single click anywhere in your VS solutionTests can be ran with a single click anywhere in your VS solution Supports 'Debugger' context to step into a unit testSupports 'Debugger' context to step into a unit test For developers who want to work in the Visual Studio IDE there is the add-onFor developers who want to work in the Visual Studio IDE there is the add-on Tests can be ran with a single click anywhere in your VS solutionTests can be ran with a single click anywhere in your VS solution Supports 'Debugger' context to step into a unit testSupports 'Debugger' context to step into a unit test

NUnitNUnit Live Demo

Unit Testing Best Practices

Naming Standards for Unit Tests Test name should express a specific requirementTest name should express a specific requirement Test name should includeTest name should include Expected input or stateExpected input or state Expected result output or stateExpected result output or state Name of the tested method or className of the tested method or class Test name should express a specific requirementTest name should express a specific requirement Test name should includeTest name should include Expected input or stateExpected input or state Expected result output or stateExpected result output or state Name of the tested method or className of the tested method or class

Naming Standards for Unit Tests – Example Given the method:Given the method: with requirement to ignore numbers > 100 in the summing process The test name should be:The test name should be:Sum_NumberIgnoredIfBiggerThan100 Given the method:Given the method: with requirement to ignore numbers > 100 in the summing process The test name should be:The test name should be:Sum_NumberIgnoredIfBiggerThan100 public int Sum(params int[] values)

When Should a Test be Changed or Removed? Generally, a passing test should never be removedGenerally, a passing test should never be removed These tests make sure that code changes don’t break working codeThese tests make sure that code changes don’t break working code A passing test should only be changed to make it more readableA passing test should only be changed to make it more readable When failing tests don’t pass, it usually means there are conflicting requirementsWhen failing tests don’t pass, it usually means there are conflicting requirements Generally, a passing test should never be removedGenerally, a passing test should never be removed These tests make sure that code changes don’t break working codeThese tests make sure that code changes don’t break working code A passing test should only be changed to make it more readableA passing test should only be changed to make it more readable When failing tests don’t pass, it usually means there are conflicting requirementsWhen failing tests don’t pass, it usually means there are conflicting requirements

When Should a Test be Changed or Removed? (2) Example:Example: New features allows negative numbersNew features allows negative numbers Example:Example: New features allows negative numbersNew features allows negative numbers [ExpectedException(typeof(Exception), "Negatives not allowed")] void Sum_FirstNegativeNumberThrowsException() { Sum (-1,1,2); Sum (-1,1,2);}

New developer writes the following test:New developer writes the following test: Earlier test fails due to a requirement changeEarlier test fails due to a requirement change New developer writes the following test:New developer writes the following test: Earlier test fails due to a requirement changeEarlier test fails due to a requirement change When Should a Test be Changed or Removed? (3) void Sum_FirstNegativeNumberCalculatesCorrectly() { int sumResult = sum(-1, 1, 2); int sumResult = sum(-1, 1, 2); Assert.AreEqual(2, sumResult); Assert.AreEqual(2, sumResult);}

Two course of actions:Two course of actions: 1.Delete the failing test after verifying if it’s valid 2.Change the old test: Either testing the new requirementEither testing the new requirement Or test the older requirement under new settingsOr test the older requirement under new settings Two course of actions:Two course of actions: 1.Delete the failing test after verifying if it’s valid 2.Change the old test: Either testing the new requirementEither testing the new requirement Or test the older requirement under new settingsOr test the older requirement under new settings When Should a Test be Changed or Removed? (4)

Tests Should Reflect Required Reality What’s wrong with the following test?What’s wrong with the following test? A failing test should prove that there is something wrong with the production codeA failing test should prove that there is something wrong with the production code Not with the unit test codeNot with the unit test code What’s wrong with the following test?What’s wrong with the following test? A failing test should prove that there is something wrong with the production codeA failing test should prove that there is something wrong with the production code Not with the unit test codeNot with the unit test code int Sum(int a, int b) –> returns sum of a and b public void Sum_AddsOneAndTwo() { int result = Sum(1,2); int result = Sum(1,2); Assert.AreEqual(4, result, "Bad sum"); Assert.AreEqual(4, result, "Bad sum");}

What Should Assert Messages Say? Assert message in a test is one of the most important thingsAssert message in a test is one of the most important things Tells us what we expected to happen but didn’t, and what happened insteadTells us what we expected to happen but didn’t, and what happened instead Good assert message helps us track bugs and understand unit tests more easilyGood assert message helps us track bugs and understand unit tests more easily Assert message in a test is one of the most important thingsAssert message in a test is one of the most important things Tells us what we expected to happen but didn’t, and what happened insteadTells us what we expected to happen but didn’t, and what happened instead Good assert message helps us track bugs and understand unit tests more easilyGood assert message helps us track bugs and understand unit tests more easily

Express what should have happened and what did not happenExpress what should have happened and what did not happen “ Verify() did not throw any exception”“ Verify() did not throw any exception” “ Connect() did not open the connection before returning it”“ Connect() did not open the connection before returning it” Do not:Do not: Provide empty or meaningless messagesProvide empty or meaningless messages Provide messages that repeat the name of the test caseProvide messages that repeat the name of the test case Express what should have happened and what did not happenExpress what should have happened and what did not happen “ Verify() did not throw any exception”“ Verify() did not throw any exception” “ Connect() did not open the connection before returning it”“ Connect() did not open the connection before returning it” Do not:Do not: Provide empty or meaningless messagesProvide empty or meaningless messages Provide messages that repeat the name of the test caseProvide messages that repeat the name of the test case What Should Assert Messages Say? (2)

Avoid Multiple Asserts in a Single Unit Test Disadvantages of multiple asserts in one test case:Disadvantages of multiple asserts in one test case: If the first assert fails, the test execution stops for this test caseIf the first assert fails, the test execution stops for this test case Affect future coders to add assertions to test rather than introducing a new oneAffect future coders to add assertions to test rather than introducing a new one void Sum_AnyParamBiggerThan1000IsNotSummed() { Assert.AreEqual(3, Sum(1001, 1, 2); Assert.AreEqual(3, Sum(1, 1001, 2); Assert.AreEqual(3, Sum(1, 2, 1001); }

Mock Objects Usage Q: When should mock objects be used? A: Mock objects are used when one needs to replace or remove dependencies from code under test Example:Example: Class LoginManager manages user logins with the following responsibility:Class LoginManager manages user logins with the following responsibility: When login fails, class reports to a logger classWhen login fails, class reports to a logger class

Mock Objects Usage (2) The unit test should test the class logic without having to configure or rely on the availability of the loggerThe unit test should test the class logic without having to configure or rely on the availability of the logger So we replace the logger class with a "fake" oneSo we replace the logger class with a "fake" one

Unit Testing – The Challenge The concept of Unit Testing has been around for many yearsThe concept of Unit Testing has been around for many years New methodologies in particular XP, have turned unit testing into a cardinal foundation of software developmentNew methodologies in particular XP, have turned unit testing into a cardinal foundation of software development Writing good & effective Unit Tests is hard!Writing good & effective Unit Tests is hard! This is where supporting integrated tools and suggested guidelines enter the pictureThis is where supporting integrated tools and suggested guidelines enter the picture The ultimate goal is tools that generate unit tests automaticallyThe ultimate goal is tools that generate unit tests automatically The concept of Unit Testing has been around for many yearsThe concept of Unit Testing has been around for many years New methodologies in particular XP, have turned unit testing into a cardinal foundation of software developmentNew methodologies in particular XP, have turned unit testing into a cardinal foundation of software development Writing good & effective Unit Tests is hard!Writing good & effective Unit Tests is hard! This is where supporting integrated tools and suggested guidelines enter the pictureThis is where supporting integrated tools and suggested guidelines enter the picture The ultimate goal is tools that generate unit tests automaticallyThe ultimate goal is tools that generate unit tests automatically

Useful Links NUnit – – Extreme Programming – Programming – XP Programming – Programming – Advanced Unit Testing– Unit Testing– Continuous Integration – egration.htmlContinuous Integration – egration.html egration.html egration.html NUnit – – Extreme Programming – Programming – XP Programming – Programming – Advanced Unit Testing– Unit Testing– Continuous Integration – egration.htmlContinuous Integration – egration.html egration.html egration.html

Unit Testing Fundamentals and NUnit Questions?Questions?

ExercisesExercises 1. Write two classes: Student and Course. Students should have name and faculty number. Name can not be empty and Faculty number is between and Each course contain set of students. Students in a course should be less than 30 and can join and leave courses.

Exercises (2) 2.Write NUnit tests for these two classes Use 2 class library projects in Visual Studio: School.csproj and TestSchool.csprojUse 2 class library projects in Visual Studio: School.csproj and TestSchool.csproj 3.Execute the tests using GUI and console variants of NUnit 2.Write NUnit tests for these two classes Use 2 class library projects in Visual Studio: School.csproj and TestSchool.csprojUse 2 class library projects in Visual Studio: School.csproj and TestSchool.csproj 3.Execute the tests using GUI and console variants of NUnit