1. Indexes Rose-Hulman Institute of Technology Curt Clifton

2. Overview  Introduction to Indexes  Index Architecture  How SQL Server Retrieves Stored Data  How SQL Server Maintains Index and Heap Structures  Deciding Which Columns to Index

3. Data Pages Page 7 Page 8Page 9 Akhtar Funk Smith Martin... Page 4Page 5Page 6... Con Funk White... Rudd White Barr... Smith Ota Jones... Martin Phua Jones Smith... Ganio Jones Hall... Storing and Accessing Data  How Data Is Stored Rows are stored in data pages Heaps are a collection of data pages for a table

4. Whether to Create Indexes  Why to Create an Index Speeds up data access Enforces uniqueness of rows  Why Not to Create an Index Consumes disk space Incurs overhead

5. Using Heaps  SQL Server:  Uses Index Allocation Map Pages That: Contain information on where the extents of a heap are stored Navigate through the heap and find available space for new rows being inserted Connect data pages  Reclaims Space for New Rows in the Heap When a Row Is Deleted

6. Using Clustered Indexes  Each Table Can Have Only One Clustered Index  The Physical Row Order of the Table and the Order of Rows in the Index Are the Same  Key Value Uniqueness Is Maintained Explicitly or Implicitly

7. Using Nonclustered Indexes  Nonclustered Indexes Are the SQL Server Default  Existing Nonclustered Indexes Are Automatically Rebuilt When: An existing clustered index is dropped A new clustered index is created The DROP_EXISTING option is used to change which columns define the clustered index

8. Maintaining Index and Heap Structures

9. Page Splits in an Index Index Pages Non-Leaf Level INSERT member (last name) VALUES lastname = ‘Jackson' INSERT member (last name) VALUES lastname = ‘Jackson' Lang Smith … … Akhtar Ganio … … Akhtar … … Martin Jackson Leaf Level (Key Value) Akhtar Barr Borm Buhl … … … … … … … … … … Lang Martin Moris … … … … … … … … … … Smith … … … … … … … … … … ……… Ganio Hall Hart Jones … … … … … … … … … … … Leaf Level (Key Value) Akhtar Barr Borm Buhl … … … … … … … … … … Jackson Jones … … … … … … Lang Martin Moris … … … … … … … … … … Smith … … … … … … … … … … ……… Ganio Hall Hart … … … … … … … Jackson…

10. Forwarding Pointer in a Heap Non-Leaf Level Page 12 - Root Page 37Page 28 Leaf Level (Key Value) Page 41Page 51Page 61Page 71 Akhtar... Martin Akhtar Barr Con Funk 4:706:01 4:705:03 4:704:01 4:706:02 4:704:02 Martin Smith... Smith White 4:706:03 4:708:04 4:707:01 4:704:03 4:705:02 Akhtar Ganio... Ganio Hall Jones 4:709:01 4:709:04 4:709:02 4:708:03 4:707:03 Heap Page 707 Page 708Page 709 01 02 03 04... Akhtar Funk Smith Martin... Page 704Page 705Page 706 01 02 03... Conn Funk White... 01 02 03... Rudd White Barr... 01 02 03 04 05... Smith Ota Jones Corets Nash 01 02 03 04... Martin Phua Jones Smith... 01 02 03... Ganio Jones Hall... Martin Ota Phua Rudd 4:708:01 4:706:04 4:707:02 4:708:02 4:705:01 Non Clustered Index File ID #4 id indid = 2 root sysindexes Non-Leaf Level Page 12 - Root Page 37Page 28 Leaf Level (Key Value) Page 41Page 51Page 61Page 71 Akhtar... Martin Akhtar Barr Con Funk 4:706:01 4:705:03 4:704:01 4:706:02 4:704:02 Martin Smith... Smith White 4:706:03 4:708:04 4:707:01 4:704:03 4:705:02 Akhtar Ganio... Ganio Hall Jones 4:709:01 4:709:04 4:709:02 4:708:03 4:707:03 Heap Page 707 Page 708Page 709 01 02 03 04... Akhtar Funk Smith Martin... Page 704Page 705Page 706 01 02 03... Conn Funk White... 01 02 03... Rudd White Barr... 01 02 03 04 05... Smith Ota Jones Corets Nash 01 02 03 04... Martin Phua Jones Smith... 01 02 03... Ganio Jones Hall... Martin Ota Phua Rudd 4:708:01 4:706:04 4:707:02 4:708:02 4:705:01 Non clustered Index File ID #4 02...Ota 02...Ota Martin Martin 02Ota 04...Ota Ota4:707:02 UPDATE member SET Address = WHERE lastname = 'Ota' UPDATE member SET Address = WHERE lastname = 'Ota'

11. Row Updates  Generally do not cause rows to move  Like a delete followed by an insert Logically Sometimes practically  Batch updates touch each index once

12. Deletion  Deletion creates “ghost records”  Reclaiming space Free pages when empty For indexed table:  Can overwrite ghost records immediately For non-indexed table:  Compact records when more space is needed for insert

13. Deciding What to Index

14. What You Need to Know  Logical and Physical Database Design  Data Characteristics  How Data Is Used The types of queries performed The frequency of queries that are typically performed

15. Indexing Guidelines  Columns to Index Primary and foreign keys Those frequently searched in ranges Those frequently accessed in sorted order Those frequently grouped together during aggregation  Columns Not to Index Those seldom referenced in queries Those that contain few unique values Those defined with text, ntext, or image data types

16. Choosing the Clustered Index  Heavily Updated Tables A clustered index with an identity column keeps updated pages in memory  Sorting A clustered index keeps the data pre-sorted  Column Length and Data Type Limit the number of columns Reduce the number of characters Use the smallest data type possible

17. Data Characteristics – Densitylast_namelast_namefirst_namefirst_name Randall...... Joshua Randall Cynthia Randall Tristan...... Ota Lani...... SELECT * FROM member WHERE last_name = ‘Ota’ Low Density SELECT * FROM member WHERE last_name = ‘Randall’ High Density

18. Data Characteristics – Selectivity  How effective is a column at selecting a subset of the data  A property of a given query: Rows matching property / Total number of rows

19. Determining Selectivity High selectivity member_nomember_no 1 1 2 2.... last_namelast_namefirst_namefirst_name Randall Flood Joshua Kathie.. 10000 Anderson Bill SELECT * FROM member WHERE member_no > 8999 1000 10000 = 10% Number of rows meeting criteria Total number of rows in table = Low selectivity member_nomember_no 1 1 2 2.... last_namelast_namefirst_namefirst_name Randall Flood Joshua Kathie.. 10000 Anderson Bill SELECT * FROM member WHERE member_no < 9001 9000 10000 = 90% Number of rows meeting criteria Total number of rows in table =

20. Indexing to Support Queries  Writing Good Search Arguments Specify a WHERE clause in the query Verify that the WHERE clause limits the number of rows Avoid using leading wildcards

21. Introduction to Statistics

22. Standard Distribution of Values F - JA - EK - OP - UV - Z Last Name Number of Last Names Even Distribution of Values Last Name Number of Last Names C - FA - BG - K L - NO - Z

23. How Statistics Are Gathered  DMBS reads/samples column values Produces an evenly distributed sorted list of values  Performs a full scan or sampling of rows Depending on size of table and granularity wanted  Selects samplings if necessary Picks rows to be sampled Includes all rows on the page of selected rows

24. Creating Statistics  Automatically Creating Statistics Indexed columns that contain data Non-indexed columns that are used in a join predicate or a WHERE clause  Manually Creating Statistics Columns that are not indexed All columns other than the first column of a composite index

25. Viewing Statistics  The DBCC SHOW_STATISTICS Statement Returns Statistical Information in the Distribution Page for an Index or Column  Statistical Information Includes: The time when the statistics were last updated The number of rows sampled to produce the histogram Density information Average key length Histogram step information

26. Performance Considerations  Create Indexes on Foreign Keys  Create the Clustered Index Before Nonclustered Indexes  Consider Creating Composite Indexes  Create Multiple Indexes for a Table That Is Read Frequently