© 2012 Autodesk Automated Testing with the AutoCAD ®.NET API Scott McFarlane Senior Software Engineer, Woolpert, Inc.

© 2012 Autodesk Class Summary Automated testing, which includes unit testing and integration testing, is one of the cornerstones of the Agile development process. Yet we see it as difficult and expensive, especially for programs written with the AutoCAD.NET API. So what makes it so difficult? What are the obstacles? Can we overcome them? This class will explore the topic of automated testing, specifically for.NET programs written for products based on AutoCAD software. You will learn how good programming practices, such as single responsibility, abstraction, interface development, and dependency inversion, go hand in hand with writing testable code, and how to use these techniques to decouple your code from the AutoCAD API, enabling you to unit test your business logic in Microsoft® Visual Studio®. We’ll also look at a few of the popular testing tools, including the Gallio Automation Platform, which integrates AutoCAD into its test runner.

© 2012 Autodesk Learning Objectives At the end of this class, you will be able to:  Recognize the obstacles to doing software testing, particularly with programs written for AutoCAD  Understand the goals, philosophy and principles of test automation  Appreciate the benefits of test automation  Use the correct Terminology: What is a mock, a stub, a fake, a dummy?  Use abstraction and dependency inversion to write testable code

© 2012 Autodesk Automated Testing

© 2012 Autodesk Why Am I Not Doing Automated Testing?  My programs always work correctly  I never need to modify or refactor my code  I enjoy spending hours tracking down bugs  I haven’t yet realized the benefits  I haven’t learned to do it well

© 2012 Autodesk Code-Driven Testing Frameworks  The growing family of code-driven testing frameworks have come to be known collectively as xUnit  Based on a design by Kent Beck, originally implemented for Smalltalk as SUnit, then ported to Java as JUnit.  Wikipedia lists over 360 testing frameworks for over 70 different languages and environments.

© 2012 Autodesk Basic xUnit Architecture  Test Case  Also called Testcase Class  Implemented as an interface, abstract base class, or class attribute  Contains one or more Test Methods  Typically shares the same Test Fixture  Test Method  Contains the logic for each test

© 2012 Autodesk Basic xUnit Architecture  Test Suite  A set of tests that share the same Test Fixture  SUT (System Under Test)  The actual class/method we are testing  Test Fixture  All the things we need to have in place to run a test  4-Step Test Execution  1. Setup  2. Exercise  3. Verify  4. Teardown

© 2012 Autodesk Topics  The Goals of Test Automation  Philosophy and Principles of Test Automation  Terminology: What is a mock, a stub, a fake, a dummy?

© 2012 Autodesk

Goals of Test Automation  Tests should help us improve quality  Tests should help us understand the system under test (SUT)  Tests should reduce (and -not introduce) risk  Tests should be easy to run  Tests should be easy to write and maintain  Tests should require minimal maintenance as the system evolves around them

© 2012 Autodesk Tests Should Help Us Improve Quality  What is Quality?  1. Is the software built correctly?  2. Have we built the correct software?  Tests act as an executable specification  Tests help prevent bugs  Tests help localize bugs  But… Quality is not just testing “Trying to improve the quality of software by doing more testing is like trying to lose weight by weighing yourself more often.” - Steve McConnell

© 2012 Autodesk Tests Should Help Us Understand the SUT  Rather than pouring over the code to answer:  What is this code supposed to do?  What would the result be if... ?  Tests should show the test reader how the code is supposed to work.  Tests are less likely to become out-of-sync with code than documentation

© 2012 Autodesk Tests Should Reduce (Not Introduce) Risk  Tests provide a safety net for incremental development and refactoring  Without tests, we are reluctant to make changes or try different approaches – risk is too high.  Writing good tests is not easy – It takes lots of practice  Keep tests simple, clean, easy to read  Don’t be afraid to use Test Utility Methods.  Dangers (Risks) of Automated Testing  Putting test logic in production code  Testing too many things at once  Testing too little

© 2012 Autodesk Tests Should Be Easy to Run  Tests must be…  Fully automated  Self-checking  Independent  Repeatable  Fast

© 2012 Autodesk Tests Should Require Minimal Maintenance as the System Evolves Around Them  Test only public interfaces and outcomes when possible  Avoid testing private methods directly (implementation details)  Use Mocks only when necessary  Minimize duplicate code in tests  Tests should facilitate (not hinder) our ability to make changes to the production code.

© 2012 Autodesk Summary of Goals  Trustworthiness  Are you able to run your test and trust the results?  Maintainability  Treat your test code with the same care as your production code.  Readability  Can you look at a test and quickly understand what is being tested and why it failed?

© 2012 Autodesk Testing Philosophy  Test first versus test after  Test-by-test versus test all-at-once  Outside-in versus inside-out  Behavior verification versus state verification

© 2012 Autodesk Definitions (Meszaros)  Test Double – Any object that is used in place of a real object for the purposes of testing. Mocks, stubs, fakes and dummies are all test doubles.  Mock – an object that can monitor the indirect outputs (behavior) of the SUT. Your tests assert against the mock object (rather than the SUT itself) to see if it has passed.  Stub – an object that can control indirect inputs into the SUT. A test can use multiple subs, but can only have one mock object.  Fake – an object used by the SUT that behaves exactly like the real object, but may be used to isolate the SUT from external resources. An in-memory database is an example of a Test Double.  Dummy – an object that is required by the SUT to run, but plays no active role in the test.

© 2012 Autodesk Behavior Verification vs. State Verification  Behavior Verification  State Verification Class under test Mock Test Communicate Assert Class under test Stub Test Communicate Assert Dependent Component

© 2012 Autodesk Unit Tests vs. Integration Tests Unit Tests  Should run w/o config  Isolated  No teardown required  Run in memory  Fast  Repeatable  Run often Integration Tests  May require config  Use external resources  May require teardown  May use disk or database  Slow  May not be repeatable  Run less often

© 2012 Autodesk Unit Tests vs. Integration Tests  Do not mix - Keep in separate projects/assemblies

© 2012 Autodesk Unit Testing “Thorns”  MessageBox.Show()  Timer  WebClient  System.IO (File, etc.)  DateTime.Now  Singletons/Statics

© 2012 Autodesk Testing Tools  Testing Framework (xUnit)  MSTest  NUnit  MbUnit/Gallio  Isolation Framework  NSubstitute  Moq  FakeItEasy  Other  Fluent Assertions  NCrunch

© 2012 Autodesk Obstacles to Testing AutoCAD Code

© 2012 Autodesk Obstacles to Testing AutoCAD Code  Our programs are hosted by AutoCAD  Heavy user interaction  Tightly coupled with AutoCAD API

© 2012 Autodesk Example

© 2012 Autodesk Abstraction and Dependency Injection

© 2012 Autodesk Why is Abstraction Important?  The Problem  Classes often contain dependencies on other classes to do their work.  If a class makes assumptions about how its dependent classes are implemented, the class becomes difficult to reuse in combination with other components.  Such classes are also very difficult to unit test because they cannot be isolated from their dependencies.

© 2012 Autodesk Why is Abstraction Important?  The Solution  Establish a common set of protocols by which classes interact, separately from the classes themselves.  Promotes the use of software design patterns (particularly dependency injection) that result in code that is more testable, extensible, maintainable, scalable, and reusable.

© 2012 Autodesk Typical Application Business Logic Logger FILE User Interface Exception Handler Data Access Web Service Client Component Configuration

© 2012 Autodesk Business Logic User Interface Logger FILE Exception Handler Data Access Web Service Client Component Configuration IDataRepository ILogger IMessageBox IProgressBar IExceptionHandler ISomeWebService Interface IConfiguration IHostApplication Host Application Typical Application

© 2012 Autodesk Business Logic Test Double IDataRepository ILogger IMessageBox IProgressBar IExceptionHandler ISomeWebService Interface IConfiguration Test Double IHostApplication Typical Application

© 2012 Autodesk Example 1 public class Example1 { public void DoTheWork() { DataRepository dataRepository = new DataRepository(); Logger logger = new Logger(); logger.Log("Getting the data"); DataSet theData = dataRepository.GetSomeData(); // Do some work with the data... logger.Log("Done."); }

© 2012 Autodesk Example 2 public class Example2 { private readonly IDataRepository _dataRepository; private readonly ILogger _logger; public Example2(IDataRepository dataRepository, ILogger logger) { _dataRepository = dataRepository; _logger = logger; } public void DoTheWork() { _logger.Log("Getting the data"); DataSet theData = _dataRepository.GetSomeData(); // Do some work with the data... _logger.Log("Done."); }

© 2012 Autodesk Software Engineering Principles  Separation of Concerns – This class is now only responsible for the specific job it was designed to do.  Abstraction – By using interfaces, we have established a set of protocols by which the components interact, separately from the classes themselves.  Inversion of Control – The class has relinquished control of the creation and initialization of its dependencies.  Dependency Injection – This pattern is based on Inversion of Control, and describes the way in which an object obtains references to its dependencies.

© 2012 Autodesk Application Architecture AutoCAD Main Plug-in Code (Bootstrapper) AutoCAD Specific Logic Business Logic AcDbMgd.dll AcCoreMgd.dll AcMgd.dll Shared Abstractions User Interface Other Dependencies Logging Configuration Databases Web Services Etc… Depends on AutoCAD API No dependency on AutoCAD API Interfaces Domain Objects Implements AcCoreConsole.exe Implements Unit Tests

© 2012 Autodesk Example

© 2012 Autodesk Autodesk, AutoCAD* [*if/when mentioned in the pertinent material, followed by an alphabetical list of all other trademarks mentioned in the material] are registered trademarks or trademarks of Autodesk, Inc., and/or its subsidiaries and/or affiliates in the USA and/or other countries. All other brand names, product names, or trademarks belong to their respective holders. Autodesk reserves the right to alter product and services offerings, and specifications and pricing at any time without notice, and is not responsible for typographical or graphical errors that may appear in this document. © 2012 Autodesk, Inc. All rights reserved.