Automation Testing with Visual Studio Team System Tester Edition Wely (wely_live@live.com) Microsoft MVP, Visual C# http://netindonesia.net/blogs/wely

Agenda Review of VSTS Objective of VSTS Test Edition Test Type Available in VSTS Web Testing Load Testing Manual Testing Generic Testing Ordered Testing

Visual Studio Team System 2008 Team Suite Visual Studio Industry Partners for 3rd party products Architecture Edition Database Edition Development Edition Unit Test Code Coverage Code Analysis Profiling Code Metrics Test Edition Performance Testing Manual Testing Web Test Load Test Ordered Test Application Modeling Database Deployment Infrastructure and Deployment Modeling Database Change Mgmt. Database Testing MSF Process and Guidance Our Focus Today Visual Studio Professional Edition Team Explorer Test Load Agent Team Explorer Team Foundation Server Version Control Reporting and BI Team Build Work Item Tracking Team Portal © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Objective of VSTS – Test Edition 6/1/2018 6:41 PM Objective of VSTS – Test Edition Mitigate Business Risk Make better informed go-live decisions Catch performance bugs before going live Bring Quality earlier in lifecycle Find bugs before going live Ensure developers can test prior to handing off to QA Consistent tooling Better collaboration across dev/test Ensure effective triage Enables testing of the solution through the lifecycle Test authoring Web (HTTP/HTTPS), AJAX, Unit Running tests Scalability & Performance Testing Manual Testing Publishing test reports Integrates tightly with the Application Lifecycle Management (ALM) features of VSTS/TFS for effective team collaboration Provides traceability between tests and other artifacts Source code Work Items Build Management MICROSOFT CONFIDENTIAL © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Test Types Available in VSTS TechEd 2004 6/1/2018 6:41 PM Test Types Available in VSTS Test Type Description Available In… Unit Tests code that exercises your project’s functions/methods VS Pro and above Web Tests Capture and playback of http-level interactions, including AJAX Test Edition Load Tests Simulate multiple users running your automated tests Manual Tests Step through those tasks you’ve not automated Database Tests SQL Server 2000 and 2005 database logic validation Database Edition, Test Edition Other Generic (command line) tests Performance Tests Detailed metrics on code performance, memory, critical paths, etc Developer Edition © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Web Testing Uses a recorder for test creation: Records web requests using Internet Explorer Analyzes the HTML traffic– each response can be validated for expected behavior Ordered series of HTTP requests against a target web applications The test could be run / automated on Any browsers (IE, Firefox, Safari, etc) Any technology / languages (ASP.NET, JSP, PHP, etc) Offline or online Web Tests are of two types: Recorded Tests Coded Tests

Prepare Web Application Web Testing (cont’d) Process of Web Testing Usage Analyzing the details of the tested web Regression Testing Automation Data Entry But beware of spamming  For further usage of other tests (eg: performance testing, load testing) Prepare Web Application Record Web Test Insert Rules / Checks Run Web Test Analyze result

Demo Web Test Add comment Add validation rule Generate and modify code Use data source Used positively: data insertion Used negatively: spam  (demo with wordpress) Profiling

Load Testing Load testing is the process of putting demand on a system or device and measuring its response Primary goal of a Load Test is to simulate many users accessing server at the same time Load Tests are used in several different types of testing: Smoke: How your application performs under light loads for short duration Stress: To determine if the application will run successfully for a sustained duration under heavy load Performance: To determine how responsive your application is Capacity: To determine how many users and/or transactions a given system will support and still meet performance goals

Identify Key Scenarios Load Testing Process For example: Login to the site. Browse albums. Register to the site. Response Time Throughput Resource Utilization Maximum User Load Identify Objectives Identify Key Scenarios Identify Workload Identify Metrics Create Test Cases Simulate Load Analyze the Results Scenario — Placing a order Total number of concurrent users at normal load condition — 500 users # of users for the scenario — every 3 minutes incrementally add 100 users Test duration — 1 hours Think time - Random think time between 1 and 10 seconds

Demo Load Test Think time Threshold (memory and processor) Summary (avg page per second) Each page response time (makes sense?) Analyzing test result Multi-view (table, graph) Export Result

Manual Test A manual test is a test performed by a human being. Captured in a text document or a Microsoft Word document that lists the steps. Use manual tests when other test types would be too difficult or impossible to create or run. Capture pass/fail result

Demo Manual Test

Generic Test Use a generic test to wrap an existing test, program, or third-party tool that behaves as follows: It can be run from a command line. It returns a value of Pass or Fail. Optionally, it also returns detailed results for 'inner' tests, tests that it contains.

Demo Generic Test

Ordered Test An ordered test contains other tests that are meant to be run in a specified order. An ordered test appears as a single test in the Test Manager and Test View windows. Its results appear in a single row in the Test Results window.

Demo Ordered Test

References VSTS 2008 Test Edition VSTS Video on ASP.NET http://msdn.microsoft.com/en-us/library/ms182409.aspx VSTS Video on ASP.NET www.asp.net/learn/vsts-videos/ “How Do I?” Videos for Team System http://msdn.microsoft.com/en-us/teamsystem/bb507749.aspx Jobsite Starter Kit http://www.binaryintellect.net/default.aspx Web and Load Test Tutorial http://blogs.msdn.com/amit_chatterjee/archive/2009/01/29/web-and-load-testing-with-visual-studio-team-system.aspx

Question? Wely (wely_live@live.com) Microsoft MVP, Visual C# http://netindonesia.net/blogs/wely