Building Robust and Reliable Software Jason Anderson, Microsoft

Agenda Team System Overview Static Analysis Profiling Unit Testing Test Driven Development Code Coverage Team Developer and Team Foundation Server Questions

Development Teams Infrastructure Architect Solution Architect Project Manager Developer Tester End User

Visual Studio Team System Process and Architecture Guidance Visual Studio Industry Partners Dynamic Code Analyzer Visual Studio Team Architect Static Code AnalyzerCode ProfilerUnit TestingCode CoverageVisio and UML ModelingTeam Foundation Client (includes CAL)Visual Studio Professional EditionClass DesignerLoad TestingManual TestingTest Case ManagementApplication DesignerLogical Infra. DesignerDeployment Designer Visual Studio Team Developer Visual Studio Team Test Dynamic Code AnalyzerStatic Code Analyzer Code Profiler Unit TestingCode Coverage Change ManagementWork Item TrackingReportingProject Site Visual Studio Team Foundation Integration ServicesProject ManagementBig Build

Static Analysis

Find errors in your code before you run or deploy it Checks range from style to code correctness to security issues Integrated into the Team System Build Environment

Static Analysis The dev process without Static Analysis Compile Code Binary PreprocGrammarLinkRaiseErrors

Static Analysis The dev process with Static Analysis Code Compile StaticAnalysis AnalyzeCodeCheckRulesRaiseErrors Binary

Static Analysis Managed Code Example static void Initialize(string id) { if (id == "") return; if (id == "") return; try try { Utilities.OpenUserAccount(id); Utilities.OpenUserAccount(id); } catch(Exception ex) catch(Exception ex) { // user must not exist // user must not exist Utilities.CleanAccountState(id); Utilities.CleanAccountState(id); Console.WriteLine(ex); Console.WriteLine(ex); }} Compiles Static Analysis gives warnings

Static Analysis Managed Code Example static void Initialize(string id) { if (id == "") return; if (id == "") return; try try { Utilities.OpenUserAccount(id); Utilities.OpenUserAccount(id); } catch(Exception ex) catch(Exception ex) { // user must not exist // user must not exist Utilities.CleanAccountState(id); Utilities.CleanAccountState(id); Console.WriteLine(ex); Console.WriteLine(ex); }} e:\Logger.cs(39): 'Logger.Initialize(System.String)#System.Void' Modify the following catch clauses in 'Logger.Initialize' by catching a more specific exception type or rethrowing the exception ' catch(Exception) {...}' e:\logger.cs(36): 'Logger.Initialize(System.String)#System.Void' Replace all calls to "".Equals or String.Empty.Equals in 'Logger.Initialize(System.String)' with checks for Length == 0.

Static Analysis Managed Code Example static void Initialize(string id) { if (id.Length == 0) if (id.Length == 0) { throw new ArgumentExeception(“Bad A…”); } try try { Utilities.OpenUserAccount(id); Utilities.OpenUserAccount(id); } catch(Exception ex) catch(Exception ex) { // user must not exist // user must not exist Utilities.CleanAccountState(id); Utilities.CleanAccountState(id); throw ex; throw ex; }} Improve my code

Static Analysis

Profiling

Profiling Bad performance is expensive ● Customer perception ● Hardware ● Investigation

Profiling Instrumentation main AMethod BMethod main 001 AMethod0023 BMethod0098 … Adds instructions to your code to monitor

Profiling Sampling main AMethod BMethod Analyze without changing your app main 3 AMethod 6 BMethod 9 2

Profiler Able to do both managed and unmanaged Able to do both instrumentation and sampling Used by many internal teams and on customer engagements

Profiling

Unit Testing

Unit tests are tests written by programmers to check the functionality of production code ● Always automated ● Generally one-to-one mapping between Unit Test and production code Typical problems: ● Too much overhead for everyone to learn ● Too tedious ● Isn’t effective enough to warrant the cost ● Not process agnostic

Writing a Unit Test made Easy Low overhead, very few moving parts We do provide simple helper APIs and easy to understand attributes We don’t force inheritance, interfaces, etc

Unit Test Code Generation Human error sometimes accounts for missed methods and classes Code Generation can develop the skeleton of your unit tests ● Namespaces ● Classes ● Methods Generated code fails with result ‘Inconclusive’

Unit Testing

Test Driven Development

Think about the code before you write it, have a failing test before you write product code The Test Case is a specification ● User Scenarios ● Business Cases ● Minimal bar, boundary conditions, performance, etc Infrastructure can be painful ● Build Errors ● Managed your failures

Test Driven Development

Code Coverage

Test Authoring isn’t about quantity, it’s about quality Code coverage helps you monitor your tests effectiveness ● Team Members can analyze results at a high- level ● Can also analyze source for specific missed methods and branches

Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

Code Coverage

Team Developer and Team Foundation Server

Visual Studio Team System Process and Architecture Guidance Visual Studio Industry Partners Dynamic Code Analyzer Visual Studio Team Architect Static Code AnalyzerCode ProfilerUnit TestingCode CoverageVisio and UML ModelingTeam Foundation Client (includes CAL)Visual Studio Professional EditionClass DesignerLoad TestingManual TestingTest Case ManagementApplication DesignerLogical Infra. DesignerDeployment Designer Visual Studio Team Developer Visual Studio Team Test Dynamic Code AnalyzerStatic Code Analyzer Code Profiler Unit TestingCode Coverage Change ManagementWork Item TrackingReportingProject Site Visual Studio Team Foundation Integration ServicesProject ManagementBig Build

Visual Studio Team System Process and Architecture Guidance Visual Studio Industry Partners Change ManagementWork Item TrackingReportingProject SiteIntegration ServicesProject ManagementDynamic Code Analyzer Visual Studio Team Architect Static Code AnalyzerCode ProfilerUnit TestingCode CoverageVisio and UML ModelingTeam Foundation Client (includes CAL)Visual Studio Professional EditionLoad TestingManual TestingTest Case ManagementApplication DesignerLogical Infra. DesignerDeployment Designer Visual Studio Team Developer Visual Studio Team Test Change ManagementWork Item Tracking Reporting Visual Studio Team Foundation Class Designer Big Build

Roles Addressed by Team System Project ManagerArchitectDeveloperTester Change ManagementWork Item TrackingReportingProject Portal Visual Studio Team Foundation Integration ServicesProject Management VisualStudioIDE

Work Item Integration Project Management can assign work items and track across lifecycle of the project Developers and Testers can assign defects and these integrate with PM work items Artifacts (tests, product modules, source code) can be associated with Work Items

Reviews Find / Fix Rates…

..and Daily Trends

…and Bugs Against Code Churn and Testing Activity

Team Developer Static Analysis Profiling Unit Testing Code Coverage AND MORE!!! Integrated with TFS ● Source Code Control ● Project Management ● Bug Tracking ● Reports All fully functional from the VSTS 2005

© 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.