1. DAT430 Extending SQL Server 2000 Functionality with User-Defined Functions: Hidden Tricks Fernando G. Guerrero S olid Q uality L earning fernando@solidqualitylearning.com

2. Perhaps you heard about UDFs somewhere Perhaps you know how to convert stored procedures into UDFs Or to created parameterized views (called now Inline UDFs) Or to extend SQL Server builtin function with your own scalar UDFs Is there anything else you could do with them?

3. Quick info about Fernando (2 milliseconds) MCSD, MCSE+Internet (W2K), MCDBA, MCT, SQL Server MVP CEO and Principal Mentor at Solid Quality Learning Writing for SQL Sever Magazine and SQL Server Professional Co-author of Microsoft SQL Server 2000 Programming by Example (ISBN : 0789724499) Author of the.NET Operations Guide and the.NET Deployment Guide for Microsoft TechNet

4. S olid Q uality L earning (3 ms) An association of SQL Server experts from around the world Principal Mentors: Itzik Ben-Gan Kalen Delaney Fernando G. Guerrero Michael Hotek Brian Moran Ron Talmage Kimberly L. Tripp Helping you get the best out of your SQL Server: Training Consulting Mentoring Stay tuned to http://www.SolidQualityLearning.com

5. Agenda Bypassing UDFs limitations Some undocumented UDFs you should know about How to create a system UDF How to redefine system stored procedures as system UDFs Combining UDFs with Indexed views Indexing computed columns based on UDFs Modifying data through UDFs

6. A little hidden UDF gem: fn_dblog demo demo

7. Bypassing UDFs Limitations When a user defined UDF is deterministic? Can we build non-deterministic UDFs? Can use non-deterministic functions inside a UDF?

8. When is a user-defined UDF deterministic? Every function, built-in or user-defined, referenced in the function is deterministic. The function is defined with the SCHEMABINDING option. The function does not execute extended stored procedures. You can use built-in non-deterministic functions inside a UDF, even if BOL tells you that it is not possibleUDFBOL

9. Using non-deterministic functions inside a UDF If a function is not valid inside a UDF and you really need to use it there: Use a RowSet function: OPENROWSET OPENQUERY OPENDATASOURCE Examples: Generate random numbers from a UDF Using Date functions inside a UDF

10. Limitations’ workarounds and some tricks for Scalar UDF How to get random numbers from a UDF How to use dates inside UDF Recursion in scalar UDF How to modify actual data from inside a UDF Let’s look at them in more detail!

11. How to get random numbers from a UDF RAND is not allowed inside a UDF Get time information from sysprocesses (last_batch) Only meaningful for user connections Returns service start time for system processes (such as Lazy Writer) Not very good approach Make it more complex combining some trig or log functions (such as PI) Use OPENQUERY to call back and everything will be allowed…… including RAND !?!?!?!

12. How to use dates inside UDF Date related functions are not allowed inside a UDF Get time information from sysprocesses (last_batch), as before, but it isn’t reliable Use OPENQUERY or OPENROWSET to call back and everything will be allowed… including GetDate !?!?!?! including GetDate !?!?!?! Why date functions are not allowed inside UDFs? Perhaps someone at MS hates them

13. Recursion in scalar UDF Nothing wrong with it but… I would try a different solution Limited to 32 levels Could be useful to navigate binary trees A valid alternative to recursive stored procedure that retrieve data Example: The employees tree The employees tree

14. How to modify actual data from inside a UDF It is not allowed… by definition But if you can use ADO objects from VB to connect to SQL Server and modify data… You can do the same thing from UDFs with sp_OAxxx procedures However… I don’t see the need for it: I would always use stored procedures to modify SQL Server data But it is possible, as in this example, as in this example

15. Inserting with OPENQUERY? OPENQUERY is designed to retrieve data from any OLEDB Provider But it actually execute batches of instructions OPENQUERY makes and OLEDB connection back to SQL Server Let’s see what happens what happens

16. Can you “execute” a UDF? UDF style SELECT dbo.TotalPrice(12, 0.2, 25.4) Stored procedure style EXECUTE @Total = TotalPrice 12, @Discount = 0.2, @UnitPrice = 25.4 Some code Great for dealing with optional parameters, but you lose the entire reason of using a UDF

17. Using table variables inside UDFs Valid inside Scalar and Table valued UDFs This is the only case where you can use DML functions inside a UDF Not valid for Inline UDFs I’ll show you an exampleexample

18. Consuming XML data inside UDFs Consuming XML data inside a UDF requires execution of: sp_xml_preparedocument sp_xml_removedocument The CREATE UDF works very well But it can’t be used: Server: Msg 557, Level 16, State 2, Procedure GetProductsFromXML, Line 12 Only functions and extended stored procedures can be executed from within a function. I have an exampleexample

19. Returning XML data from UDFs The FOR XML clause is not valid inside a UDF However, you might think about using OPENQUERY: SELECT * FROM OPENQUERY(CLOCKWORK, 'SELECT TOP 5 * FROM Northwind.dbo.Products FOR XML AUTO') This query returns an image column, and this cannot be used as result of a scalar UDF The result of a scalar UDF is nothing else than a special OUTPUT parameter of the query plan Yes, I have an exampleexample

20. Some undocumented Scalar UDFs Where to find them? $SQLInstallDir$\Install\Replsys.sql $SQLInstallDir$\Install\Procsyst.sql Some examples on how to use themSome examples on how to use them: fn_chariswhitespace fn_replquotename fn_replmakestringliteral fn_mssharedversion Mostly defined for internal use by replication procedures I didn’t find any real use of them, but some definitions are very interesting

21. A Documented System Inline UDF ::fn_trace_gettable(@filename, @numfiles) Returns trace file information in a table format. @filename is the trace physical file path @numfiles is the number of rollover files to read (Default to all files) SELECT * FROM ::fn_trace_gettable( 'c:\my_trace.trc', default)

22. And an Undocumented System Inline UDF ::fn_dblog(@start, @end) Reads the Transaction Log @start is the starting Log Sequence Number (LSN) @end is the last LSN Use (NULL, NULL) to read the entire log Be careful with the @start and @end format!!!! Convert them first from HEX:HEX:HEX into DEC:DEC:DEC SELECT TOP 10 [Current LSN], Operation FROM ::fn_dblog(NULL, NULL) ORDER BY [Current LSN] DESC

23. Joining UDFs UDF to UDF SELECT CompanyName, OrderID, TotalValue FROM dbo.OrdersByValue(10000) AS OBV JOIN dbo.CustomersByCountry(‘USA’) C ON OBV.CustomerID = C.CustomerID UDF to Table SELECT CompanyName, OrderID, TotalValue FROM dbo.OrdersByValue(10000) AS OBV JOIN Customers C ON OBV.CustomerID = C.CustomerID Some codecode

24. Creating system UDFs Must be created in the master database SQL Server must be configured to allow updates to system tables The owner must be system_function_schema That’s it! Very useful for converting system stored procedures into system UDFs: If you’d like to use a system procedure in the FROM clause And the procedure contains valid UDF code

25. Example: Converting sp_lock into fn_lock fn_lock can provide you better functionality than sp_lock: Search for waiting processes Search for locks affecting a given object Display locks affecting a given connection Display locks affecting the same resource Look at the code For example: SELECT * FROM ::fn_lock() WHERE Status = 'WAIT‘ You can convert sp_who into fn_who as well

26. Example: Getting Windows users and groups information fn_getntgroupmembers gives you the members of a Windows group fn_getntgroups gives you the groups that a given user belong too Look at the code For example: SELECT * FROM ::fn_getntgroups(‘MyDomain\MyUser')

27. UDFs in constraint definitions Using scalar UDFs as a DEFAULT definitions Using scalar UDFs in CHECK constraints Using scalar UDFs in PRIMARY KEY fields Using scalar UDFs in computed columns “UDFs Provide a New Identity” (SQL Server Magazine, March 2001): http://www.sqlmag.com/Articles/Index.cfm?ArticleID=16354 Look at the code: ALTER TABLE IDTest ADD CONSTRAINT def_IDTest DEFAULT (dbo.fn_GetNewID(OBJECT_ID('IDTest'))) FOR ID

28. Combining UDFs with Indexed views Create the view first, as complex as you’d need Create an index on the view by using the searchable column Create an InLine UDF selecting data from this view Use the searchable column as parameter of the UDF This provides efficient way of using this indexed view Look at this exampleexample

29. Indexing computed columns based on UDFs Create a computed column based on: A UDF A combination of UDFs The result needs to be precise and deterministic Create an index on this column in the usual way Only useful when: the UDF is complex to run and its result only changes from time to time Many reading operations using this expression as a searchable argument Look at this exampleexample

30. Modifying data through UDFs Only valid for InLine UDFs They are almost views, so they give you similar functionality Can you modify actual data through views? As long as the view is not read-only Or you define an INSTEAD of trigger for it Can you create INSTEAD OF triggers on a UDF? NO But you can create a UDF on top of a view that has INSTEAD OF triggers defined 

31. Inserting actual data through UDFs INSERT dbo.GetCustomersFromUSA() (CustomerID, CompanyName, Country) VALUES ('ZZZZZ', 'Dummy Customer', 'USA') Let’s take a look at the query planquery plan

32. Deleting actual data through UDFs DELETE dbo.GetCustomersFromUSA() WHERE CustomerID = 'ZZZZZ' Let’s take a look at the query planquery plan

33. Updating actual data through UDFs UPDATE dbo.GetCustomersFromUSA() SET CompanyName = 'New Customer' WHERE CustomerID = 'ZZZZZ' Let’s take a look at the query planquery plan

34. Summary You can bypass some UDFs limitations You can learn to love some undocumented UDFs You know you can create a system UDF And redefine system stored procedures as system UDFs And get extra performance and flexibility by combining UDFs with Indexed views And you can modifying data through UDFs Isn’t this cool?

35. Ask The Experts Get Your Questions Answered I’ll be at the Ask the Experts area: 2 July 2003: 12:00-14:00 3 July 2003: 11:00-13:00 4 July 2003: 12:00-14:00 You’ll see me around some more times

36. Community Resources http://www.microsoft.com/communities/default.mspx Most Valuable Professional (MVP) http://www.mvp.support.microsoft.com/ Newsgroups Converse online with Microsoft Newsgroups, including Worldwide http://www.microsoft.com/communities/newsgroups/default.mspx User Groups Meet and learn with your peers http://www.microsoft.com/communities/usergroups/default.mspx

37. Thank you! Questions? Download the source code of this session from: http://www.solidqualitylearning.com/conferences You can contact me at: fernando@solidqualitylearning.com

38. evaluations evaluations

39. © 2003 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.