1. Enabling Better Software through Better Testing Jason Anderson, Microsoft jasand@microsoft.com

2. Agenda Team System Overview Test Authoring & Execution ● Manual Testing ● Web Testing ● Automated \ Unit Testing ● Load Testing Test Case Management Code Coverage Test Extensibility Team Tester and Team Foundation Server Questions

3. Development Teams Infrastructure Architect Solution Architect Project Manager Developer Tester End User

4. Visual Studio Team System Process and Architecture Guidance Visual Studio Industry Partners Dynamic Code Analyzer Visual Studio Team Architect Static Code AnalyzerCode ProfilerUnit TestingCode CoverageVisio and UML ModelingTeam Foundation Client (includes CAL)Visual Studio Professional EditionLoad TestingManual TestingTest Case ManagementApplication DesignerLogical Infra. DesignerDeployment Designer Visual Studio Team Developer Visual Studio Team Test Load TestingManual Testing Test Case Management Change ManagementWork Item TrackingReportingProject Site Visual Studio Team Foundation Integration ServicesProject ManagementBig BuildClass Designer

5. Test Authoring

6. QA SKU UI: Authoring Tests Design Principal: Enable authoring features specific to testing without impeding standard development experience Primary UI Elements ● Solution Explorer ● Test View Largest Real-estate reserved for code ● Result details can share space via tab selection with explicit user gesture

7. Types Explained VisualStudioIDE EDT QA Tools Manual Unit Generic Web Load ASP Ordered 3 rd Party

8. Test Execution

9. QA SKU UI: Execution Design Principal: Provide rich environment for management and control of test cases Primary UI Elements ● Test Manager ● Command Line Parser ● Results Window Primary user scenario is not test author, rather to manage and execute tests Local or Remote Execution ● Distributed Across Machines ● With Machine Constraints

10. Writing and Executing Tests

11. Test Case Management

12. Fully Integrated with Visual Studio Uses a local file that can be checked into source code control and shared ● Can be used without the project system ● Versioning Test Lists to managed Test Cases ● Tests in one or more Test Lists ● Test Lists can be hierarchical ● Can be used from the Command Line

13. Test Case Management

14. Code Coverage

15. Test Authoring isn’t about quantity, it’s about quality Code coverage helps you monitor your tests effectiveness ● Team Members can analyze results at a high- level ● Can also analyze source for specific missed methods and branches

16. Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

17. Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

18. Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

19. Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

20. Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

21. Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

22. Code Coverage

23. Test Extensibility

24. Why Test Extensibility Must not lose investment in current infrastructure Testing requirements are diverse Integration beyond Microsoft

25. Extensibility Story Generic Tests ● Requires minimal integration work ● Limitations to Integration with VS Extended Test Types ● Custom type for full Test Type extensibility ● Means of adding authoring, execution, and reporting extensibility ● Custom types are 1 st class test citizens

26. Generic Tests How execution works GenericTestType Existing Test Metadata VisualStudioIDE ExecutionProcess YourTestProcess GenericTestType Metadata

27. Generic Tests How execution works VisualStudioIDE ExecutionProcess YourTestProcess YourTest YourTest YourTest

28. Test Type Extensibility Extended Types Test Environment can be extended to include your test tools Authoring, Execution, Results plug-ins Offers first class test type Used internally to create test types Test Type Extensibility Framework MS Test Types VSIP Test Types

29. Test Type Extensibility How execution works VisualStudioIDE ExecutionProcess Test Adaptor Test Test Updates Results Details Viewer Run Config Settings

30. Test View Your Test Source Your Test Case Your Test Result Summary

31. Your Details

32. Your Test Case Test Explorer

33. Custom Editor / Wizard Test Items Management UI General Results View Custom Detailed Results View Generics Test Type Hook into Run Config Settings Config Settings Context-sensitive Right-click Commands Complete control of Test item execution

34. Team Tester and Team Foundation Server

35. Visual Studio Team System Process and Architecture Guidance Visual Studio Industry Partners Dynamic Code Analyzer Visual Studio Team Architect Static Code AnalyzerCode ProfilerUnit TestingCode CoverageVisio and UML ModelingTeam Foundation Client (includes CAL)Visual Studio Professional EditionLoad TestingManual TestingTest Case ManagementApplication DesignerLogical Infra. DesignerDeployment Designer Visual Studio Team Developer Visual Studio Team Test Load TestingManual Testing Test Case Management Change ManagementWork Item TrackingReportingProject Site Visual Studio Team Foundation Integration ServicesProject ManagementBig BuildClass Designer

36. Visual Studio Team System Process and Architecture Guidance Visual Studio Industry Partners Change ManagementWork Item TrackingReportingProject SiteIntegration ServicesProject ManagementDynamic Code Analyzer Visual Studio Team Architect Static Code AnalyzerCode ProfilerUnit TestingCode CoverageVisio and UML ModelingTeam Foundation Client (includes CAL)Visual Studio Professional EditionLoad TestingManual TestingTest Case ManagementApplication DesignerLogical Infra. DesignerDeployment Designer Visual Studio Team Developer Visual Studio Team Test Change ManagementWork Item Tracking Reporting Visual Studio Team Foundation Class Designer Big Build

37. Roles Addressed by Team System Project ManagerArchitectDeveloperTester Change ManagementWork Item TrackingReportingProject Portal Visual Studio Team Foundation Integration ServicesProject Management VisualStudioIDE

38. Work Item Integration Project Management can assign work items and track across lifecycle of the project Testers and Developers can assign defects and these integrate with PM work items Artifacts (tests, product modules, source code) can be associated with Work Items

40. Reporting Integration Information aggregated from Team System services used in reporting Stored in central database for trending Drill-down into operational stores for more detail

41. Reviews Find / Fix Rates…

42. ..and Daily Trends

43. …and Bugs Against Code Churn and Testing Activity

44. Visual Studio Team System Team Test Write your Tests with or without Code Manage and Execute Test Cases Extend the Visual Studio Test Types Integrated with TFS ● Source Code Control ● Project Management ● Bug Tracking ● Reports All fully functional from the Visual Studio Team System 2005

45. © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.