1. SQL Server Data Collector From Every Angle
Gianluca Sartori
sqlconsulting.it

2. Sponsors & Media Partners

3. Organizers

4. Gianluca Sartori Independent SQL Server consultant
Works with SQL Server since version 7
MCTS, MCITP, MCT
Scuderia Ferrari
Blog: spaghettidba.com
Twitter: @spaghettidba

5. Agenda What is the Data Collector Why Data Collection? How it works
Troubleshooting
Baselining
Monitoring
How it works
SQL Server Data Collector at work
Setup
Reports
Querying
Extending

6. What is the Data Collector?
Framework and tools for:
Low overhead collection
Persistence of performance data
Diagnostic analysis
Monitoring
Reporting

7. Why Data Collection? Goals: Troubleshooting
What happened tonight at 03:40?
Tune performance
Which queries consume most resources?
Improve management
How fast are my databases growing?
Monitoring
Is the system behaving normally?

8. Baselining What is a baseline?
Describes performance under “normal” conditions
Useful for:
Comparing performance
Monitoring
Troubleshooting
Trend Analysis and Capacity Planning

9. Challenges What to collect? How to collect? How often?
Where to persist?
Which retention policy?
How to analyze?

10. What to collect? Performance indicators Dynamic Management Objects
DMV
DMF
Performance Counters
Query Stats
Wait Stats

11. Dynamic Management Objects
sys.dm_os_ OS information
sys.dm_io_ Disk performance
sys.dm_exec_ Sessions, requests, connections
sys.dm_db_ Database info
sys.dm_tran_ Transactions, locks
sys.dm_ Audits, broker, CDC, FileStream, FTS, HADR, Repl, RG, XE…
Which DMOs?
SQL Server 2012 contains 141 DMVs and DMFs!

12. Performance Counters CPU Memory Disk SQL - specific … Which counters?
SQL Server 2012 exposes more than 400 counters!

13. Challenges What to collect  System Collection Sets
How to collect  Data Collector Infrastructure
How often  Default Schedules
Where to persist  Management Datawarehouse
Which retention policy  Cleanup jobs
How to analyze  Canned reports

14. Data Collector Overview
Data Collection UI
(Object Explorer)
Management Data
Warehouse
Data Collection Configuration
Collection Set Reports
Target
Data Collector
Target
Data Collector

15. Components Data Collector Extensible data collection infrastructure
Low overhead (approx. 5% CPU overhead)
Includes system collection sets
Supports multiple data providers
Support for SQL Server relational engine only
other services can be added through extension

16. Components Management Data Warehouse
Data repository for baseline and historical comparisons
Central database for multiple SQL Server instances
Size for around 300 Mb per day per instance
Automatically purges old data
Aggregated “olystic” repository for Utility Control Point

17. Components Management Studio components:
Data collection configuration UI
Setup
Properties
Configuration
Data collector reports
Canned reports based on system collection sets
Entry point for monitoring and troubleshooting

18. Concepts Data Provider Collector Type Collection Item Collection Set
Source of information
Collector Type
Reads data from data providers
Collection Item
Instance of a collector type
Data input, frequency and target table
Collection Set
Group of collection items
Logic unit of data collection

19. Data Collector (dcexec.exe)
Architecture
msdb
SSMS
SQL Agent
SQL Server
MDW
Data Collector (dcexec.exe)
Operating System
Data Collector Cache

20. Architecture Collected Collection Job Data (SSIS runtime)
Data Collector Cache
MDW
Upload Job
(SSIS runtime)

21. DEMO
Identifying Data Collector components

22. Data Collector SSMS UI Setup Configuring collection sets
Create a Management Datawarehouse
Set up data collection
Configuring collection sets
Starting and stopping
Collect and upload
Configure properties
View log

23. DEMO
Setting up data collection

24. Data Collector Reports
System reports
Based on system collection sets
Server Activity
Performance dashboard
Many drill down reports
Query Statistics
Most expensive queries
Drill down
Disk Usage
Monitor growth trends

25. DEMO
Using system reports

26. Management Datawarehouse
Schemas:
core
Internal usage
snapshots
Data gathered by system collection sets
custom_snapshots
Data gathered by custom collection sets

27. DEMO
Querying the Management Datawarehouse:
Who was active?

28. Extending the Data Collector
Custom collection sets
Based on generic T-SQL collector
Some limitations apply
Based on custom SSIS packages
Complicated

29. DEMO
Creating a custom collection set

30. Extending the Data Collector
Custom reports
Integrate in SSMS
Distribute to all DBAs?
Deploy to Reporting Services
Bill Ramos’ MDW reports
Analyze with PowerBI tools
PowerQuery
PowerView

31. DEMO
Using Bill Ramos’ MDW reports

32. Bottom line
Data collection provides out-of-the-box tools to collect and analyze performance data
Free - Ships with SQL Server
Some limitations apply
Steep learning curve
Viable alternative to commercial monitoring tools when money is a constraint
Always test first!

33. Q&A
Questions?

34. Resources Bill Ramos – MDW Reports Bart Duncan – Query Hash Statistics
Bart Duncan – Query Hash Statistics

35. #sqlsat264 #sqlsatancona
Thanks!