1. Too Slow?: Use VS2010 Profiling & Load Testing to Manage Performance Issues Benjamin Day

2. Consultant, Coach, Trainer Scrum.org Classes  Professional Scrum Developer (PSD)  Professional Scrum Foundations (PSF) TechEd, VSLive, DevTeach, O’Reilly OSCON Visual Studio Magazine, Redmond Developer News Microsoft MVP for Visual Studio ALM Team Foundation Server, TDD, Testing Best Practices, Silverlight, Windows Azure http://blog.benday.com benday@benday.com

3. Thanks. To my content reviewers  Jerri Chiu, Microsoft  Dennis Bass, Microsoft

4. The Agenda Basics of  Web Performance Tests  Load Tests  Load Test Rigs  Performance Explorer Existing application  Identify performance problems  Go from bad performance to good  ASP.NET & WCF

5. Disclaimer: I’m going to use the term “load testing” imprecisely…a lot.

6. Here we go.

7. The Purpose of Load Testing Identify the capabilities of your application Possible goals  Explore  Verify  Find the limits  Crush

8. Best Practice: The Obligatory “Eat Your Vegetables” Load test throughout the development cycle. Catch problems early Establish performance baseline Watch for performance trends Cheaper to fix early Plan for performance

9. Cheaper To Fix Early

10. Dose Of Reality You know you should work that way. You probably don’t. This talk is for you. You have problems. How do you figure out what’s going wrong?

11. Inspiration for this talk Real customer engagement Spent years developing a web application Released it Seriously grumpy customers Huge hardware ~20 simultaneous users Fixed it  4000+ simultaneous users  Reduced hardware

12. The Tools Visual Studio Ultimate Web Performance Tests Load Tests Load Test Rigs Visual Studio Premium Profiling Tools Performance Wizard

13. TOUR OF THE APPLICATION demo

14. Disaster Strikes. The application is deployed to production. Users are losing their minds.  Intermittent errors on the site.  Site is slow. Your boss is seriously angry.  “Fix it!”

15. The Problem. “No Repro”  “Works on my box.”  You don’t know what the errors are.  The site seems to work fine. Wild guess: it dies under load. You need to create some load.

16. The Plan. 1.Script user activity with Web Performance Tests 2.Simulate a mix of users using Load Tests 3.Simulate lots of users with a Load Test Rig 4.Reproduce the errors...hopefully 5.Find code problems with the Performance Wizard 6.Fix the code

17. Web Performance Tests.

18. The Plan. 1.Script user activity with Web Performance Tests 2.Simulate a mix of users using Load Tests 3.Simulate lots of users with a Load Test Rig 4.Reproduce the errors...hopefully 5.Find code problems with the Performance Wizard 6.Fix the code

19. What is a Web Performance Test? Test type in Visual Studio 2010 Ultimate Simulate a user using an ASP.NET application  HTTP traffic Building block of Load Tests

20. Tip: Web Tests & User Stories Web Tests should simulate a User Story Example:  As an administrator, I need to search for a person by username so that I can deactivate that person’s account.  As a customer, I want to order a pizza and pay with a credit card. Helps to organize your suite

21. CREATE A BASIC WEB TEST demo

22. More about Web Tests Collection of web requests Not for Test-First Development Recorded or coded Can be data-driven Make them do real stuff with  Validation Rules  Extraction Rules

23. Tip: Use Web Tests as a Quality Gate to QA Web Tests are great for “smoke testing” Run against a build to learn if the build is worth QA-ing Save QA a lot of time  (They like this.)

24. Parameterization Minimize hard coded values Use test context values in requests and validations Web server addresses for requests Double “squiggle bracket” syntax  {{variable_name}} Essential for data-driven web tests

25. Data-driven Web Tests Run web test once per row in data source Simulate multiple users doing *similar* actions Data source types  OLE DB, CSV, XML Parameterization Better tests, better coverage

26. WEB TESTS IN THE UNDERPERFORMING APPLICATION demo

27. Tip: Wait for stability Wait until your screens are reasonably stable Control name changes  broken web tests Control hierarchy changes  name changes  broken web tests

28. Load Tests.

29. The Plan. 1.Script user activity with Web Performance Tests 2.Simulate a mix of users using Load Tests 3.Simulate lots of users with a Load Test Rig 4.Reproduce the errors...hopefully 5.Find code problems with the Performance Wizard 6.Fix the code

30. What are Load Tests? Mix of Web Performance and Unit Tests Simulates a mix of users doing a mix of things Simulate  Network speeds  Different browsers  Varying user activity loads

31. Types of “Load Tests” Performance Test  Response time  Time lapses  Duration Load Test  Behavior under normal/peak workload  Actual workload Stress Test  Surfacing issues under extreme conditions and resource failures Source: J.D. Meier, http://blogs.msdn.com/b/jmeier/archive/2007/03/21/performance-vs-load-vs-stress-testing.aspx

32. Why would you put a Unit Test in a Load Test? Helpful for testing WCF Cumbersome to simulate WCF via Web Test  (You’d have to re-invent the wheel.) Unit tests can re-use the WCF libraries  (Done and done.) Important for Silverlight and SOA http://wcfloadtest.codeplex.com/ http://sqlloadtest.codeplex.com/

33. HEY KIDS, LET’S CREATE SOME LOAD TESTS! demo (…and then you all say, “Hooray!”)

34. Running Load Tests.

35. The Plan. 1.Script user activity with Web Performance Tests 2.Simulate a mix of users using Load Tests 3.Simulate lots of users with a Load Test Rig 4.Reproduce the errors...hopefully 5.Find code problems with the Performance Wizard 6.Fix the code

36. Two ways to run load tests. Generate load from Visual Studio 2010  Uses only 1 core/processor  Limited to 250 simulated users Generate load from a Test Rig  Far better test  Simulate lots more users

37. Load Test Agent Licensing Microsoft Visual Studio Team System 2010 Test Load Virtual User Pack Simulate up to 1000 virtual users Used to cost $$$

38. Visual Studio 2010 Load Test Feature Pack Used to cost ~$3500.00 per 1000 simulated users Free starting March 8, 2011 Visual Studio 2010 Ultimate with MSDN Simulate unlimited amount of load  (Limited only by your hardware.) http://www.microsoft.com/visualstudio/ en-us/products/2010-editions/load-test-virtual-user-pack

39. Load Test Rig & The Application Controller  Orchestrates  Licensing  Perf Mon Counters Agent (1..n)  Simulates users

40. Configure the Controller Install SQL Express  Optional  Stores collected test run data Install Visual Studio 2010 Test Controller Run the process as an ActiveDirectory domain user  Example: LoadTestController Add a Virtual User License Pack

41. Configure the Agent(s) Install the Agent Installation will ask which Controller Tip: Run the Agent service(s) using an Active Directory user  Example: LoadAgent

42. Permissions Performance Monitor Users  Controller user needs to be in this group  Typical servers: Controller, Agent(s), Web Server, Database server TeamTestControllerAdmins  Run tests on the rig  View results  Clean up the results database TeamTestControllerUsers  Run tests on the rig  View Results

43. The Plan. 1.Script user activity with Web Performance Tests 2.Simulate a mix of users using Load Tests 3.Simulate lots of users with a Load Test Rig 4.Reproduce the errors...hopefully 5.Find code problems with the Performance Wizard 6.Fix the code

44. RUN LOAD TESTS ON THE RIG demo

45. Hopefully, I remembered to mention the following… The “Manage Test Controllers…” dialog *.TestRunConfig settings Running the load tests Could spot the exceptions that were happening Fix the data access problems Demonstrated improved performance Compare two Load Test Runs

46. The Core Problem.

47. ‘sp_who2’ says lots of open connections.

48. Performance Profiling

49. The Plan. 1.Script user activity with Web Performance Tests 2.Simulate a mix of users using Load Tests 3.Simulate lots of users with a Load Test Rig 4.Reproduce the errors...hopefully 5.Find code problems with the Performance Wizard 6.Fix the code

50. Profiling Tools Load Tests help you find issues in a system  “Surface” issues Profiling helps you find issues in the code Unit Tests, Web Tests, Load Tests  Repeatable  Good for targeted tuning System under “natural” load  Not repeatable  Looking for rough patterns

51. Profiling Methods Gathers information at intervals (clock cycles) No code modifications Use this for first pass explorations Sampling Modifies your code Gathers detailed timing and count information Instrumentation Multi-threaded code profiling Concurrency Object creation Garbage collection.NET Memory Interaction between your application and SQL Server via ADO.NET Tier Interaction

52. Load Test Performance Sessions Via Wizard Must be only Web Performance Tests Can’t profile via Load Test Rig ASP.NET must be running in x86 mode

53. LET’S GO PROFILE SOME STUFF. demo

54. The Plan. 1.Script user activity with Web Performance Tests 2.Simulate a mix of users using Load Tests 3.Simulate lots of users with a Load Test Rig 4.Reproduce the errors...hopefully 5.Find code problems with the Performance Wizard 6.Fix the code

55. Any last questions?

56. For More Information: Articles http://tinyurl.com/3xjqgvz http:// tinyurl.com/2ulvvvr

57. For More Information: Microsoft Links Visual Studio Performance Testing Quick Reference Guide http://vsptqrg.codeplex.com/ Content Index for Web Tests & Load Tests http://tinyurl.com/ye97e52

58. For More Information: Microsoft Bloggers VS Team Test Blog Site http://blogs.msdn.com/b/vstsqualitytools/ Ed Glas - http://blogs.msdn.com/edglas/ Bill Barnett - http://blogs.msdn.com/billbar/ Sean Lumley - http://blogs.msdn.com/slumley/ Dennis Stone - http://blogs.msdn.com/densto/ Mike Taute - http://blogs.msdn.com/mtaute/ Neelesh Kamkolkar - http://blogs.msdn.com/b/nkamkolkar/

59. Thank you. blog.benday.com | www.benday.com | benday@benday.com

60. Additional information that I practically never have time to cover.

61. Miscellaneous Tips.

62. Include Load Tests In Your Nightly Build Create a Test List (*.vsmdi) Include the Test List in the build Captures trend data at 24 hours intervals Compare the load test data with Excel

63. Static Code Analysis / FxCop Help you to find issues in your code Show you IDispose errors Rules are configurable Use your judgment about which are relevant for your team

64. Avoid Static Methods & CA1822 Static Code Analysis Rule CA1822 is evil. Sacrifices maintainability for perfomance. Blog post: “Static Methods Are A Code Smell” http://bit.ly/cnEsuV

65. Only optimize known performance problems. You’d be surprised by what *ISN’T* a performance problem Don’t spend a ton of time coding fancy solutions to performance problems you *THINK* you might have Find and fix *REAL* problems

66. Tip: WCF Message Size Data structure formatting can make a huge difference Example: Banks and the States they operate in Option 1: Dictionary  Key = Bank Id  Value = State Abbreviation Option 2: Dictionary  Key = State Abbreviation  Value = Array of Bank Id’s Option 2 was approximately 90% smaller

67. Tip: Add Performance Counters Create Performance Counters (perfmon) in your app Do this early in the development cycle More directed profiling of your app during Load Tests Good for managing your app when it’s in production  (Operations people love this.)

68. Using Performance Sessions on x64 You can’t profile x64 IIS processes from the IDE Option #1: Profile from the command line  Start ASP.NET profiling using VSPerfAspNetCmd.exe  Run the web test Option #2: Run the ASP.NET process in x86 mode

69. How To Run ASP.NET in x86 Mode Set “Enable 32-bit Applications” to true on your application’s AppPool