Mission critical application testing with Distributed Replay Joe Yong Senior Program Manager

Distributed Replay What is Distributed Replay? Multi-client scalability to drive higher throughput Higher workload replay fidelity Useful for stress testing production workload or performing application compatibility testing before upgrades Why an enhanced replay tool? Use multiple low-cost clients (workstations) to handle large workload traces within reasonable time Address limitations of current SQL Trace based tools (Profiler, Upgrade Assistant) Provide enhanced features Groundwork for future workload testing capabilities

Distributed Replay basic concept QA / Production Test System change Trace File Capture Replay (before) Replay (after) Comparison Reports

Distributed Replay data & workflow Batch Request Preprocess 1 Batch Request Trace File IR file IR file IR file IR file IR file Batch Request User Controller … SQL Server Replay 2 Batch Request Multiple Replay Clients

Distributed Replay Concepts Connect time: time delta between trace start and login is established Think time: time delta between two events on the same connection

Preprocess & Replay Steps What Happens During Preprocess: Sort the events in ascending order of EventSequence Trim away events and columns not used for replay Generate an intermediate file Goal: preprocess the trace once, and intermediate file can be replayed with different options What Happens During Replay: User specified the replay options, including which clients to use Shred the intermediate file into chunks in a load-balanced fashion Dispatch the file chunks to the clients Two replay modes, stress and synchronization Under stress mode, each client sends request independently Under sync mode, client needs to coordinate with controller to know when a specific event can be released (this is optimized so it is not done for every single event)

Preprocess Options IncSystemSession: Whether system session events are filtered out MaxIdleTime: cap max idle time between events

Replay options for sequence control Synchronization Mode Stress Mode StressScaleGranularity = SPID StressScaleGranularity = Connection Submit order Event submit ordering across SPIDs Event submit ordering within SPIDS Event submit ordering within connections Submit timing Submit time synchronization ConnectTimeScale (0%–100%) ThinkTimeScale (0%–100%) UseConnectionPooling (Yes | No) When to use High-fidelity replay mode, such as functional runs Use 100% connect time scale and 100% think time scale for performance runs. Use SPID granularity and reduce connect time scale and think time scale for moderate-stress runs. Use Connection granularity and reduce connect time scale and think time scale for high-stress runs. 1 2 3

Synchronization Mode Events submitted in order across SPIDS Emulate original event submit time: Capture with original think time = 5s If replay is faster; think time is increased If replay is slower; think time is decreased If replay is slower; think time is decreased

Stress Mode (SPID Granularity) Replay: StressScaleGranularity = SPID ConnectTimeScale = 10% ThinkTimeScale = 50% Capture

Stress Mode (Connection Granularity) Replay: StressScaleGranularity = Connection ConnectTimeScale = 10% ThinkTimeScale = 50% Capture

Other Replay Features (1) Added type support for: sql_variant, money, smallmoney Healthmon for deadlock detection:

Other Replay Features (2) Automatically manages database context Avoid DB mismatch between captured trace and current DB context – DR will issue a “USE DATABASE...” command if discrepancy is detected between current DB context and DB Name column in intermediate file during replay Configurable query timeout Result trace collection Execution success/failure Execution elapsed time (microsec) (On/Off) # of rows in result sets (On/Off) result set contents Supported SQL Server versions: Input Trace File Version Target SQL Server SQL Server 2005 SQL Server 2008 / 2008 R2 / 2012 SQL Server 2008 SQL Server 2008 R2 SQL Server 2008 R2 / 2012 SQL Server 2012 SQL server 2012

Performance & scalability 40 playbacks tested internally: Most run equally fast compared to SQL Profiler Some run 3x – 10x faster than SQL Profiler DReplay can scale-out when needed, SQL Profiler has no option 2 TAP trials during CTP: Largest trace replayed: 100GB Throughput achieved using 4 clients: 3000 batch requests/sec

Application compatibility testing More thorough functional and performance testing Detect potential upgrade issues not covered by Upgrade Advisor: Test ad-hoc T-SQL embedded in application Test actual T-SQL execution, not just syntactical checks Test security configuration May include OS level security settings Detect changes in query plan, duration and results Detects use of undocumented features/objects Very rare but possible: identify changes that break or alter query behavior but is not documented (yet) Not required for all applications but highly recommended for Missions/business critical applications Complex applications Application where queries are mostly client-side, middle-tier or dynamic

Application compatibility Testing QA / Production Test Database Upgrade Trace File Capture Replay (before) Replay (after) Comparison Reports

SQL Server Upgrade Assistant V2 Leverages Profiler for SQL Trace capture Leverages Distributed Replay to replay captured workload in trace file Enhanced reporting capabilities on top of SSUA V1 Enhanced user interface

Application compatibility testing Upgrade Assistant for SQL Server V2.0 powered by SQL Server 2012 Distributed Replay

Summary Key to a successful upgrade is planning & testing Tools are intended to simplify and expedite execution, not to replace proper planning Level of pre-upgrade validation/testing is typically proportionate to business criticality There is a wealth of resources available Feedback through Microsoft Connect will drive improvements in product and upgrade process/tools We are working on plans for more automation for pre-upgrade validation

Resources Distributed Replay Upgrading to SQL Server 2012 http://msdn.microsoft.com/en-us/library/ff878183(v=SQL.110).aspx Upgrading to SQL Server 2012 http://msdn.microsoft.com/en-us/library/bb677622(v=SQL.110).aspx Upgrade Advisor http://msdn.microsoft.com/en-us/library/ms144256(v=SQL.110).aspx Application compatibility testing with Upgrade Assistant http://social.technet.microsoft.com/wiki/contents/articles/sql-server-upgrade-assistant-tool-for-denali.aspx Product feedback http://connect.microsoft.com/sqlserver

Further. Forward. Faster. Learn more Visit the Microsoft Virtual Academy Free technical SQL Server training courses Get more Download SQL Server 2012 Trial Software 180-day evaluation of SQL Server 2012 Do more Participate in virtual launch activities and collect points for prizes The more points you earn, the bigger your prize could be