Dynamic SQL: Writing Efficient Queries on the Fly Ed Pollack Sr. Database Administrator Autotask

Available Now! Dynamic SQL: Applications, Performance, and Security: http://www.amazon.com/Dynamic-SQL-Applications-Performance-Security/dp/1484218108

Agenda Increasing flexibility and performance via dynamic SQL! What is dynamic SQL? Basic dynamic SQL tips & tricks List Generation Sp_executesql SQL injection Saving output Lots of applications and examples!!! Conclusion

What is Dynamic SQL? Build TSQL query using a custom-built string. Can incorporate variables, parameters, and table data. Dynamic SQL queries can be built over the course of many statements. Any string or data manipulation functions can be used. Once built, the dynamic SQL statement is executed like a stored proc. Quick Demo: Dynamic SQL basics.

Pros & Cons of Dynamic SQL Great for optional or custom searches. Allow for dynamic WHERE, GROUP BY, HAVING, TOP, ORDER BY, etc… Can speed up complex queries where only some logic is needed. Can generate large amounts of TSQL quickly & efficiently. Can easily execute TSQL across many databases or servers. When not well-written, can be very messy & hard to follow. Manage delimiters correctly, or risk SQL injection. Permissions are different than standard SQL. Unexpected input may lead to unexpected output All dynamic SQL within quotes compiles correctly…but… Not allowed in functions.

Basic Tips for Better Dynamic SQL Document thoroughly! Debugging: Use PRINT instead of EXEC to preview text before executing. Test all input use cases thoroughly. ESPECIALLY unexpected input! Break up large procs into smaller bite-size chunks. Write dynamic SQL like regular TSQL with similar spacing, indenting, etc… Always verify spacing between variables, text, and TSQL command strings. NVARCHAR vs. VARCHAR (use the correct one!) Demo: Good dynamic SQL style.

Efficiently Generating Lists Dynamic SQL can be used to quickly build lists from variables or tables. Extremely performant versus other methods such as: Iteration (WHILE, CURSOR, etc…) XML Piecemeal/stick-built methods Export…import Demo: Efficiently generating lists from table data

sp_executesql System stored proc that allows for easy execution of dynamic SQL. Requires use of NVARCHAR for command string. Accepts parameters for both inputs and outputs. Allows for execution plan reuse (if desired). Demo: sp_executesql

SQL Injection Demo: SQL Injection The use (accidental or sinister) of delimiters to break dynamic SQL queries. Replacing quotes with double-quotes is a common solution, but not good enough! Limit security to those that need access, and only to the objects they need. Always use sp_executesql when executing command strings. Never expose error messages to the end user! Use QUOTENAME for database objects to ensure they are not manipulated Add schema name to all referenced objects (whether dbo or otherwise).

Saving Output The results of a dynamic SQL select statement can be inserted into a table. Parameters can be altered when configured with the OUTPUT keyword. Temporary tables are available within dynamic SQL for read/write. Table variables and scalar variables are NOT accessible within dynamic SQL by default. Demo: Saving dynamic SQL output

Demo: FIND! Dynamic SQL can be used to generate TSQL from system views. This allows SQL Server metadata to be very efficiently queried. We can use this data as an effective search solution for an entire SQL server! Demo: FIND!

Demo: Dynamic Pivot PIVOT allows a row set to be flipped into column headers. The column names must be predefined, though! Dynamic SQL allows for an ad-hoc column list. Demo: The crazy dynamic pivot!

Demo: Generating Schema Dynamic SQL can be used to build, alter, or drop schema. Configuration tables can be used to control these actions. Removes business logic from schema into metadata. Allows a large amount of schema changes with simple control scripts. Demo: Generating data warehouse dimension tables.

Demo: Mapping FK Relationships Foreign keys can be used to recursively map database relationships. This is useful for research and understanding relational models. If foreign keys are not used, a database dictionary can be used instead. Demo: Mapping foreign key relationships.

Demo: Database Maintenance Iterating through databases, server objects, or database objects can allow for efficient database maintenance in areas such as: Backups Index maintenance MSDB maintenance Replication maintenance Demo: Database maintenance using dynamic SQL.

Conclusion Dynamic SQL can accomplish tasks that are difficult or impossible otherwise. Only use dynamic SQL when needed. Always cleanse inputs as needed by your applications. Always validate security/access and ensure it is as minimal as possible. Be a super-duper-neat-freak! Document like your life depends on it! Be creative! Dynamic SQL can elegantly solve many complex problems.

Questions???

Contact Info & Links Ed Pollack ed7@alum.rpi.edu @EdwardPollack SQL Shack SQL Server Central SQL Saturday Albany (July 29, 2017 @ UAlbany) Thank you!!!