T-SQL Window Functions in Microsoft SQL Server Denali Itzik Ben-Gan, SolidQ

Window Functions Window Functions, Described Limitations of Alternatives Windowing Concepts Window Functions Supported in SQL Server Optimization of Window Functions © 2011 Itzik Ben-Gan, SolidQ

Window Functions, Described Functions operating on a window (set) of rows defined by an OVER clause Types of functions: Ranking Aggregate Distribution SELECT empid, ordermonth, qty, SUM(qty) OVER(PARTITION BY empid ORDER BY ordermonth ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS runqty FROM Sales.EmpOrders; © 2011 Itzik Ben-Gan, SolidQ

Limitations of Alternatives Grouped queries: Force all calculations in the query to be done in context of groups Subqueries: Start from scratch Each requires separate visit to data Window functions: Starting point is underlying query’s result set (hence allowed only in SELECT and ORDER BY) Can utilize same access to data for multiple functions © 2011 Itzik Ben-Gan, SolidQ

Windowing Concepts Partitioning Ordering Framing © 2011 Itzik Ben-Gan, SolidQ

Partitioning Filters only rows with same values in partitioning elements SELECT orderid, custid, val, CAST(100. * val / SUM(val) OVER() AS NUMERIC(5, 2)) AS pctall, CAST(100. * val / SUM(val) OVER(PARTITION BY custid) AS NUMERIC(5, 2)) AS pctcust FROM Sales.OrderValues AS O1; © 2011 Itzik Ben-Gan, SolidQ

Ordering and Framing Window order clause gives meaning to framing Framing further filters rows within partition by identifying bounds Window frame units: ROWS: offset in terms of number of rows RANGE: offset in terms of values © 2011 Itzik Ben-Gan, SolidQ

Framing, ROWS SELECT empid, ordermonth, qty, SUM(qty) OVER(PARTITION BY empid ORDER BY ordermonth ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS runqty FROM Sales.EmpOrders; -- Same as: ROWS UNBOUNDED PRECEDING © 2011 Itzik Ben-Gan, SolidQ

Framing, ROWS SELECT empid, ordermonth, MAX(qty) OVER(PARTITION BY empid ORDER BY ordermonth ROWS BETWEEN 1 PRECEDING AND 1 PRECEDING) AS prvqty, qty AS curqty, ROWS BETWEEN 1 FOLLOWING AND 1 FOLLOWING) AS nxtqty, AVG(qty) OVER(PARTITION BY empid AND 1 FOLLOWING) AS avgqty FROM Sales.EmpOrders; © 2011 Itzik Ben-Gan, SolidQ

Framing, RANGE -- this example is not supported in Denali SELECT empid, ordermonth, qty, SUM(qty) OVER(PARTITION BY empid ORDER BY ordermonth RANGE BETWEEN INTERVAL '2' MONTH PRECEDING AND CURRENT ROW) AS sum3month FROM Sales.EmpOrders; SQL Server Denali supports only UNBOUNDED and CURRENT ROW delimiters When ORDER BY specified but no explicit frame, default is RANGE UNBOUNDED PRECEDING © 2011 Itzik Ben-Gan, SolidQ

Window Functions Supported in SQL Server SQL Server 2005 added initial support: Ranking calculations Aggregates with only window partitioning SQL Server Denali has more complete support: Aggregates with also window ordering and framing Offset functions: LAG, LEAD, FIRST_VALUE, LAST_VALUE Distribution functions: PERCENT_RANK, CUME_DIST, PERCENTILE_CONT, PERCENTILE_DISC © 2011 Itzik Ben-Gan, SolidQ

Examples for Aggregates -- running sum SELECT actid, tranid, val, SUM(val) OVER(PARTITION BY actid ORDER BY tranid ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS balance FROM dbo.Transactions; -- moving average AVG(val) OVER(PARTITION BY actid ROWS BETWEEN 3 PRECEDING AND CURRENT ROW) AS movavg © 2011 Itzik Ben-Gan, SolidQ

Examples for Offset -- Diff between current order value and prev/next order values SELECT custid, orderid, orderdate, val, val - LAG(val) OVER(PARTITION BY custid ORDER BY orderdate, orderid) AS diffprv, val - LEAD(val) OVER(PARTITION BY custid ORDER BY orderdate, orderid) AS diffnxt FROM Sales.OrderValues; -- Diff between current order value and first/last order values val - FIRST_VALUE(val) OVER(PARTITION BY custid ORDER BY orderdate, orderid) AS difffirst, val - LAST_VALUE(val) OVER(PARTITION BY custid ORDER BY orderdate, orderid ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) AS difflast © 2011 Itzik Ben-Gan, SolidQ

Examples for Rank Distribution rk = RANK, nr = count, np = rows precede or peer PERCENT_RANK = (rk - 1) / (nr - 1) CUME_DIST = np / nr -- Query Computing PERCENT_RANK and CUME_DIST SELECT testid, studentid, score, PERCENT_RANK() OVER(PARTITION BY testid ORDER BY score) AS percentrank, CUME_DIST() OVER(PARTITION BY testid ORDER BY score) AS cumedist FROM Stats.Scores; © 2011 Itzik Ben-Gan, SolidQ

Examples for Inverse Distribution PERCENTILE_DISC = percentile assuming discrete distribution model PERCENTILE_CONT = percentile assuming continuouse distribution model -- Query Computing PERCENTILE_DISC and PERCENTILE_CONT DECLARE @pct AS FLOAT = 0.5; SELECT testid, score, PERCENTILE_DISC(@pct) WITHIN GROUP(ORDER BY score) OVER(PARTITION BY testid) AS percentiledisc, PERCENTILE_CONT(@pct) WITHIN GROUP(ORDER BY score) OVER(PARTITION BY testid) AS percentilecont FROM Stats.Scores; © 2011 Itzik Ben-Gan, SolidQ

Optimization of Window Functions Indexing Guidelines POC Ascending partitioning values Fast Track with UNBOUNDED PRECEDING Expanding all frame rows When frame <= 4 rows When function isn’t cumulative, e.g., MIN, MAX When extreme diff between cur, top, bottom > 10,000 or using RANGE will use on-disk spool Compute two cumulative values Function is cumulative, e.g., SUM, COUNT Frame > 4 rows © 2011 Itzik Ben-Gan, SolidQ

More info… SolidQ.com, 5-day Advanced T-SQL Class InsideTSQL.com SQLMag.com © 2011 Itzik Ben-Gan, SolidQ

For attending this session and PASS SQLRally Nordic 2011, Stockholm THANK YOU! For attending this session and PASS SQLRally Nordic 2011, Stockholm