T-SQL Power! The OVER Clause: Your Key to No-Sweat Problem Solving Tech Ed North America 2010 11/18/2018 2:11 PM Required Slide SESSION CODE: DAT317 T-SQL Power! The OVER Clause: Your Key to No-Sweat Problem Solving Adam Machanic Database Consultant SQLblog © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

WHAT IS POWER?

INNOVATION SHOULD BE EASY!

About Me amachanic@gmail.com Independent SQL Server and .NET Consultant Boston, MA Author SQL Server 2008 Internals Expert SQL Server 2005 Development Conference and INETA Speaker Connections, PASS, TechEd, DevTeach, etc. Co-Founder: SQLblog.com The SQL Server Blog Spot on the Web amachanic@gmail.com

Grouping vs. Partitioning Grouping is an N1 transformation { ColA ColB Val A B 1 2 3 C D 4 5 3 rows  1 row { 2 rows  1 row

Grouping vs. Partitioning Grouping is an N1 transformation ColA ColB SUM(Val) A B 6 C D 9 { 1 row { 1 row We gain insight, but lose information

Grouping vs. Partitioning Partitioning is an NN transformation { ColA ColB Val A B 1 2 3 C D 4 5 3 rows  3 rows { 2 rows  2 rows

Grouping vs. Partitioning Partitioning is an NN transformation { ColA ColB Val SUM(Val) OVER (…) A B 1 6 2 3 C D 4 9 5 3 rows { 2 rows We gain insight, and lose nothing

Introducing the OVER Clause First available in SQL Server 2005 Can be applied to any system or user-defined aggregate for partitioning Can be applied to a new set of “windowed” functions for ordered metrics

Aggregation: The Old Way Tech Ed North America 2010 11/18/2018 2:11 PM Aggregation: The Old Way SELECT ColA, ColB, SUM(Val) FROM T1 GROUP BY ColB © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Aggregation: With OVER SELECT ColA, ColB, SUM(Val) OVER ( PARTITION BY ColB ) FROM T1

We Aren’t Required to Partition… SELECT ColA, ColB, SUM(Val) OVER() FROM T1

Multiple Partitions in One Query? Sure. SELECT ColA, ColB, SUM(Val) OVER ( PARTITION BY ColB ) AS PartitionedVal, SUM(Val) OVER() AS NonPartitionedVal FROM T1

Ranking and Numbering Functions SQL Server 2005/2008 windowing functions: ROW_NUMBER RANK DENSE_RANK NTILE These functions can be both partitioned and ordered

ROW_NUMBER Uniquely numbers rows Tech Ed North America 2010 11/18/2018 2:11 PM ROW_NUMBER Uniquely numbers rows ColA ColB ROW_NUMBER() OVER (ORDER BY ColA) A Z 1 B Y 2 C X 3 D W 4 E V 5 © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Tech Ed North America 2010 11/18/2018 2:11 PM ROW_NUMBER – Partitioned Uniquely numbers rows, restarting at each partition ColA ColB ROW_NUMBER() OVER (PARTITION BY ColA ORDER BY ColB) A X 1 Y 2 Z 3 B © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

RANK Ranks rows, skipping numbers for ties Tech Ed North America 2010 11/18/2018 2:11 PM RANK Ranks rows, skipping numbers for ties ColA ColB RANK() OVER (ORDER BY ColA) A Z 1 Y B X 3 W C V 5 © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

DENSE_RANK Ranks rows, not skipping numbers for ties Tech Ed North America 2010 11/18/2018 2:11 PM DENSE_RANK Ranks rows, not skipping numbers for ties ColA ColB DENSE_RANK() OVER (ORDER BY ColA) A Z 1 Y B X 2 W C V 3 © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

NTILE(x) Creates closest-match subgroups based on desired size “x” Tech Ed North America 2010 11/18/2018 2:11 PM NTILE(x) Creates closest-match subgroups based on desired size “x” ColA ColB NTILE(3) OVER (ORDER BY ColA) A Z 1 Y B X W 2 C V © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Under the Covers You didn’t think this was all free—right? Partitioning and ordering both require input rows to be sorted Indexes can be aligned to reduce or eliminate the need to sort All of the partitioning work is done with the Segment iterator Watch out for spools!

And Now: A Bunch of Demos!

The OVER Clause, In Depth! demo

Summary The OVER clause makes powerful T-SQL easy… …and perhaps even fun Great performance if you do a bit of work upfront Don’t forget to be creative!

Required Slide Track PMs will supply the content for this slide, which will be inserted during the final scrub. Tech Ed North America 2010 11/18/2018 2:11 PM DAT Track Scratch 2 Win Find the DAT Track Surface Table in the Yellow Section of the TLC Try your luck to win a Zune HD Simply scratch the game pieces on the DAT Track Surface Table and Match 3 Zune HDs to win © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Resources Learning Required Slide www.microsoft.com/teched Tech Ed North America 2010 11/18/2018 2:11 PM Required Slide Resources Learning Sessions On-Demand & Community Microsoft Certification & Training Resources www.microsoft.com/teched www.microsoft.com/learning Resources for IT Professionals Resources for Developers http://microsoft.com/technet http://microsoft.com/msdn © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Complete an evaluation on CommNet and enter to win! Tech Ed North America 2010 11/18/2018 2:11 PM Required Slide Complete an evaluation on CommNet and enter to win! © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Sign up for Tech·Ed 2011 and save $500 starting June 8 – June 31st http://northamerica.msteched.com/registration   You can also register at the North America 2011 kiosk located at registration Join us in Atlanta next year

Tech Ed North America 2010 11/18/2018 2:11 PM © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Required Slide Tech Ed North America 2010 11/18/2018 2:11 PM © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.