1. Tally Ho! -- Explore the Varied Uses of Tally Tables
Aaron N. Cutshall, MSCIS, MSHI
Tally Ho! -- Explore the Varied Uses of Tally Tables

2. Just who is this guy? 30+ Years Sr. Healthcare Data Architect
Past Local Group Leader
Speaker – various events
30+
Years
M.S.
Computer
Information
Systems
B.S.
Computer
Science
M.S.
Health
Informatics

3. Something to consider…
“We cannot solve our problems with the same thinking we used when we created them.”
– Albert Einstein (Theoretical Physicist)

4. Agenda What are Tally Tables? How to Use Closing Thoughts
Purposes and Uses
How To Create Them
How to Use
Find Missing Data
Break Strings
Generate Data
Closing Thoughts

5. 5 Basic Concepts of Any Programming Language
Program Structure
The overall form of a program
Interrelationships between individual components
Variable Declaration (Names & Data Types)
Looping Structures (For, While, Conditional, etc.)
Control Structures (Logic flows & branches)
Syntax (Language Rules)
** Nothing About Set-Based Operations **

6. Procedural vs Set-Based

7. Procedural Programming
Data is very abstract
Structure or organization of the data is unimportant
Separated by different layers of abstraction
Concerned with what to do and how to do it (Imperative)
Step-by-step
Statements, Loops, Object manipulation, etc.
Different logic paths depending upon data attributes
Record by record processing
Provides total control
You know exactly what it will do
Execution method will be consistent each and every time

8. Set-Based Programming
Describes what should be done (Declarative)
Indicates the operation
Identifies the conditions
Allows the engine to determine an optimal method to obtain results based upon:
Data structure, organization and volume
Statistics from previous runs
Frees developer from mechanical/specific details
Allows for changing conditions without altering query
SQL Server is optimized to select or manipulate records sets as a whole (often at the expense of procedural operations)

9. What Are Tally Tables?

10. What Are Tally Tables? A numerical sequence table
Perform set-based sequence logic
Replace loops
Improve performance
Jeff Moden
RBAR -“Row By Agonizing Row”
Avoid Processing Data Each Row at a Time
Most common performance killer in T-SQL
Promote Set-Based Operations

11. Traditional Loop Method

12. Tally Table Method

14. Hidden Loops: Scalar UDFs
A scalar UDF (User-Defined Function)
It looks set-based but executes for each result row
Turns a set-based query into a RBAR loop!
Use Inline Table Value Function instead
More efficient than Multi-line Table Value Function
Executes as a single query
Makes used of parameters
Produces a table result
Cross Apply to join ITVF for set-based operation

15. Hidden Loops: Recursion
A recursive CTE (rCTE) calls itself
It looks set-based but it’s really a hidden loop
Each level of recursion is a separate loop instance
Each loop requires the results of the previous loop
Resource intensive
Overhead to manage recursion
Consumes about 3x resources of a simple loop
Used in many parent-child relationships (Ex. Manager-Employee)

16. Hidden Loops: Recursion
Most rCTEs can be re-written in a set-based manner to use a simple while-loop
Execute base-case query; Load results into temp table
Loop for each additional level of recursion
Use join of “recursive” query and current result set
Load results into results table for use in next level
Perform loop explicitly to save resources and time
n levels of recursion uses base case plus (n-1) executions of “recursive” query

17. Hidden Loops: Explicit Cursor
Query places data in an internal structure
Fetches record one at a time
Default settings allow for forward-backward traversing and data updates
Consumes extensive resources
If required use LOCAL, FAST_FORWARD, and READ_ONLY options

18. Hidden Loops: Internal Loops
The hidden but very high speed looping effect that set-based code experiences behind the scenes
A simple SELECT "iterates" through rows but in a manner that the optimizer knows best
Example: Windowing functions such as LAG, LEAD, ROW_NUMBER, RANK, etc.

19. Set-Based Processing Does NOT mean "all in one query“
A single query can contain a hidden loop
Poor use of User-Defined Scalar Functions
Multi-Line Table Value Functions can also cause loops
Recursive CTEs contain hidden loops
CAN mean something that has a loop
Recursion can be done in a loop per recursion level
Queries can then be executed in a set-based manner
Does mean "touching" each row minimally
Does require a paradigm shift in thinking

20. What We Covered Purposes and Uses How To Create Them How to Use
Inline Table-Valued Function
Fixed Table
How to Use
Find Missing Data
Perform String Splits
Generate Data
Replace Many Types of Loops & Cursors

21. Closing Thoughts Benefits over loops & cursors: Set-Based
Improves performance
Simplifies the amount of code needed
Improved Maintainability
More concise code
Less chance of errors
Check out SQLServerCentral.com for more

22. References
Jeff Moden:
Dwain Camps:
The Code Project:
Red Gate:

23. Questions & Comments Aaron N. Cutshall BONUS:
A TON of free eBooks from Microsoft, RedGate and SentryOne!
PRESENTATION FEEDBACK:
Your thoughts needed
Improve presentations
Make this event even more valuable!!!
Aaron N. Cutshall
@sqlrv