Enabling Better Software through Better Testing Jason Anderson, Microsoft

Agenda Team System Overview Test Authoring & Execution ● Manual Testing ● Web Testing ● Automated \ Unit Testing ● Load Testing Test Case Management Code Coverage Test Extensibility Team Tester and Team Foundation Server Questions

Development Teams Infrastructure Architect Solution Architect Project Manager Developer Tester End User

Visual Studio Team System Process and Architecture Guidance Visual Studio Industry Partners Dynamic Code Analyzer Visual Studio Team Architect Static Code AnalyzerCode ProfilerUnit TestingCode CoverageVisio and UML ModelingTeam Foundation Client (includes CAL)Visual Studio Professional EditionLoad TestingManual TestingTest Case ManagementApplication DesignerLogical Infra. DesignerDeployment Designer Visual Studio Team Developer Visual Studio Team Test Load TestingManual Testing Test Case Management Change ManagementWork Item TrackingReportingProject Site Visual Studio Team Foundation Integration ServicesProject ManagementBig BuildClass Designer

Test Authoring

QA SKU UI: Authoring Tests Design Principal: Enable authoring features specific to testing without impeding standard development experience Primary UI Elements ● Solution Explorer ● Test View Largest Real-estate reserved for code ● Result details can share space via tab selection with explicit user gesture

Types Explained VisualStudioIDE EDT QA Tools Manual Unit Generic Web Load ASP Ordered 3 rd Party

Test Execution

QA SKU UI: Execution Design Principal: Provide rich environment for management and control of test cases Primary UI Elements ● Test Manager ● Command Line Parser ● Results Window Primary user scenario is not test author, rather to manage and execute tests Local or Remote Execution ● Distributed Across Machines ● With Machine Constraints

Writing and Executing Tests

Test Case Management

Fully Integrated with Visual Studio Uses a local file that can be checked into source code control and shared ● Can be used without the project system ● Versioning Test Lists to managed Test Cases ● Tests in one or more Test Lists ● Test Lists can be hierarchical ● Can be used from the Command Line

Test Case Management

Code Coverage

Test Authoring isn’t about quantity, it’s about quality Code coverage helps you monitor your tests effectiveness ● Team Members can analyze results at a high- level ● Can also analyze source for specific missed methods and branches

Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

Code Coverage

Test Extensibility

Why Test Extensibility Must not lose investment in current infrastructure Testing requirements are diverse Integration beyond Microsoft

Extensibility Story Generic Tests ● Requires minimal integration work ● Limitations to Integration with VS Extended Test Types ● Custom type for full Test Type extensibility ● Means of adding authoring, execution, and reporting extensibility ● Custom types are 1 st class test citizens

Generic Tests How execution works GenericTestType Existing Test Metadata VisualStudioIDE ExecutionProcess YourTestProcess GenericTestType Metadata

Generic Tests How execution works VisualStudioIDE ExecutionProcess YourTestProcess YourTest YourTest YourTest

Test Type Extensibility Extended Types Test Environment can be extended to include your test tools Authoring, Execution, Results plug-ins Offers first class test type Used internally to create test types Test Type Extensibility Framework MS Test Types VSIP Test Types

Test Type Extensibility How execution works VisualStudioIDE ExecutionProcess Test Adaptor Test Test Updates Results Details Viewer Run Config Settings

Test View Your Test Source Your Test Case Your Test Result Summary

Your Details

Your Test Case Test Explorer

Custom Editor / Wizard Test Items Management UI General Results View Custom Detailed Results View Generics Test Type Hook into Run Config Settings Config Settings Context-sensitive Right-click Commands Complete control of Test item execution

Team Tester and Team Foundation Server

Visual Studio Team System Process and Architecture Guidance Visual Studio Industry Partners Dynamic Code Analyzer Visual Studio Team Architect Static Code AnalyzerCode ProfilerUnit TestingCode CoverageVisio and UML ModelingTeam Foundation Client (includes CAL)Visual Studio Professional EditionLoad TestingManual TestingTest Case ManagementApplication DesignerLogical Infra. DesignerDeployment Designer Visual Studio Team Developer Visual Studio Team Test Load TestingManual Testing Test Case Management Change ManagementWork Item TrackingReportingProject Site Visual Studio Team Foundation Integration ServicesProject ManagementBig BuildClass Designer

Visual Studio Team System Process and Architecture Guidance Visual Studio Industry Partners Change ManagementWork Item TrackingReportingProject SiteIntegration ServicesProject ManagementDynamic Code Analyzer Visual Studio Team Architect Static Code AnalyzerCode ProfilerUnit TestingCode CoverageVisio and UML ModelingTeam Foundation Client (includes CAL)Visual Studio Professional EditionLoad TestingManual TestingTest Case ManagementApplication DesignerLogical Infra. DesignerDeployment Designer Visual Studio Team Developer Visual Studio Team Test Change ManagementWork Item Tracking Reporting Visual Studio Team Foundation Class Designer Big Build

Roles Addressed by Team System Project ManagerArchitectDeveloperTester Change ManagementWork Item TrackingReportingProject Portal Visual Studio Team Foundation Integration ServicesProject Management VisualStudioIDE

Work Item Integration Project Management can assign work items and track across lifecycle of the project Testers and Developers can assign defects and these integrate with PM work items Artifacts (tests, product modules, source code) can be associated with Work Items

Reporting Integration Information aggregated from Team System services used in reporting Stored in central database for trending Drill-down into operational stores for more detail

Reviews Find / Fix Rates…

..and Daily Trends

…and Bugs Against Code Churn and Testing Activity

Visual Studio Team System Team Test Write your Tests with or without Code Manage and Execute Test Cases Extend the Visual Studio Test Types Integrated with TFS ● Source Code Control ● Project Management ● Bug Tracking ● Reports All fully functional from the Visual Studio Team System 2005

© 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.