1. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | InnoDB Spatial Index Jimmy Yang Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

2. SAFE HARBOR STATEMENT The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

3. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Agenda Overview InnoDB Spatial Index as R-tree Tree Search in InnoDB Spatial Index Predicate Lock Conclusion 1 2 3 4 5

4. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Overview InnoDB now supports Spatial Index in 5.7.5  The spatial index is implemented as an R-tree, which is a special tree structure for spatial access  Can be indexed on all geometric type supported by MySQL  Supports transactions and MVCC just like other indexes in InnoDB  Isolation levels are also observed, using predicate lock preventing phantom at serializable reads

5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | InnoDB Spatial Index as R-tree R-Tree  A special data structure that used specifically for multi-dimension spatial data search.  Queries more like: find object “within”, “intersects”, “touches” another spatial object  In InnoDB, the tree is build on MySQL geometrics datatypes: POINT, LINESTRING, POLYGON, MULTIPOINT, MULTILINESTRING, MULTIPOLYGON, GEOMETRY

6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | InnoDB Spatial Index as R-tree R-tree in InnoDB  For InnoDB spatial Index, the index itself does not contain the actual data. The data is stored in the Primary Index of Table  The InnoDB spatial index consists of MBR( Minimum Bounding Box) of the object and its Primary Key.  Once the search locates the correct MBR for the queries, the real data is then retrieved from the Primary index. And the operator could apply to the actual data again for results

7. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Search in InnoDB Spatial Index Unlike B-tree search, R- tree search needs to work on multiple subtrees Search is more like a recursive in-depth search

8. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Search in InnoDB Spatial Index Use a vector to cache all qualified entries in a non- leaf page Use a “shadow” buffer page to cache all qualified rows in a leaf page Once a page is exhausted, it returns to upper lever for more qualified leaf or non-leaf page

9. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Search in InnoDB Spatial Index Entries does not have sort order across pages Entries are sorted accord (minx, miny) in page Searches are done per page (all entries in a page are qualified once). Does not support position cursor in middle of page

10. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Insertion The insertion operation begins by traversing the tree along a single path from the root to a leaf  following branches with the lowest insert penalty MBR updates on parent - if the insertion of the new leaf entry will change the leaf’s MBR, the new MBR is propagated to the parent entries by moving up the tree, until an ancestor node is encountered whose BP does not need to expand;  we also “copy” search predicates from ancestor to child nodes, if the ancestors’ predicates are consistent with the child’s updated MBR; MBR Updates Insert

11. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Insertion Use SX lock on the tree for tree insert/split, so split can happen concurrently with search Predicate locking  Replace GAP lock for phantom prevention  Check the list of predicates attached to the leaf and block on the conflicting ones until their owner transactions commit When resuming the insertion, the leaf originally located might have been split. Recognized by comparing the memorized Split Sequence Number (SSN) with the leaf’s current SSN, followed by zero or more rightlink traversals.

12. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Search/Insert synchronization with Split Sequence Number (SSN) SSN is per index, 8 bytes monotonically increase counter SSN is used to synchronize search with potential Insert Split SSN is on page header (overloads with field of FIL_PAGE_FILE_FLUSH_ LSN) Largest SSN used so far is ironed on the root page for crash recovery SSN to handle split

13. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Split Sequence Number (SSN) SSN is incremented for every split and put on the current page, the splitted right page inherits the SSN of current page Timestamp every page pointer stored on the stack with the value of the global counter as of the time the page pointer was read. 1 8 8 9 1 2 1 4 8 4 5 78 9 1 2 Page C NSN = 5 Page D NSN = 8 Page C NSN = 5 Page A NSN = 1 Page B NSN = 2 Page B NSN = 2

14. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Split Sequence Number (SSN) When search visit the node, it compares its recorded SSN with the node’s SSN. If the latter is higher, the node has been split, and then push the rightlink pointer of the node together with the originally recorded counter value on the stack. This guarantees that the right siblings split off the original node will also be examined later on. 1 8 8 9 1 2 1 4 8 4 5 78 9 1 2 Page C NSN = 5 Page D NSN = 8 Page C NSN = 5 Page A NSN = 1 Page B NSN = 2 Page B NSN = 2 Search NSN = 6

15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Deletion Key deletion is similar to B- tree in following ways:  Marked an entry as deleted for delete operation  Purge delete when it is no longer needed  Direct delete happens only with rollback and purge Node shrink  We can't simply shrink out a node, it is possibly still in a search stack  avoid incorrect pointers altogether by delaying a node deletion until there can be no more active tree operations with pointers to it A CB D

16. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | DELETION The first alternative, recovery from incorrect pointers, is impossible – the right link!  If the node has been split before it is deleted, a tree operation visiting it after the deletion may still need to traverse its rightlink. If it is deleted, the rightlink page would be lost  Repositioning within the tree by revisiting the parent is also not possible, because the parent itself might have been split or simply changed. When that happens, it is impossible to determine which nodes have been split off the deleted one. D A CB

17. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Node Shrink - Solution A node cannot be deleted while active tree operations still hold direct or indirect pointers to it Use a page lock - place a S-lock on a node if it pushes a pointer to that node onto the stack.  insert and delete operations are not prevented from physically accessing and modifying that node  A node deletion checks for S-locks by trying to acquire an X-mode lock on the respective node.

18. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Phantom Prevention and Predicate Lock No ordering for Geometry data GAP lock relies on ordering. So GAP lock does not work for R-tree Predicate Lock is used  Disadvantage on Generic Predicate lock “Generic” Predicate locks are less efficient to set and check. A search operation must set its predicate lock before the index is accessed and any data records are retrieved.

19. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Predicate Lock A hybrid system  An efficient hybrid mechanism, which associates predicate lock with tree structure  data records that are scanned, inserted or deleted are still protected by the record locking protocol  Search operations set predicate locks to prevent phantom insertions  Only insert needs to check for predicate lock (the predicate lock is inhibitive)

20. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Predicate Lock Place a predicate lock by search  If a search operation’s predicate is consistent with a node’s BP, the predicate must be attached to the node. Test on predicate lock by insertion  An insert operation can therefore limit itself to checking only the predicates attached to its target leaf. The predicates and their node attachments are only removed when the owner transaction terminates.

21. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Predicate Lock Dealing with Node Split  Predicate could be moved (if it fits for one node but not the other)  Predicate could be copied (if fits for both node MBR)  Predicate could be deleted (if it does not overlap with either MBR) Dealing with MBR expansion with insertion  Predicate need to be tested and copied from ancestor

22. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Predicate Lock Dealing with MBR expansion  Predicate need to be tested and copied from ancestor

23. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Predicate Lock-Implemenation Implementation  Different lock hash table as record hash table on lock_sys_t  Lock always on PAGE_HEAP_NO_INFIMUM (0) entry  Predicate is attached to the lock Right after lockstructure itself, after the lock entry bitmap

24. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Predicate Lock-Implemenation Implementation  Lock conflict checks both the LOCK_MODE and predicates attached  Similar to GAP lock, we have LOCK_S | LOCK_PREDICATES and LOCK_X | LOCK_PREDICATES  Locks are requalified and copied/deleted during an index split  Lock are requalified and copied to child during parent update

25. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Summary R-tree is traversed in a depth first approach with vector/stack to cached result Search and tree splits are synchronized using split sequence number (SSN) Page locks are used to prevent shrink out nodes that have active searching activity on it Predicate Locks are used to prevent phantom