1. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary: An Internal Server API That Matters Alexander Nozdrin, Principle Software Developer Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

2. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2

3. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | MySQL Community Reception @ Oracle OpenWorld Mingle with the MySQL community and the MySQL team from Oracle for a fun and informative evening! Time: September 30 (Tue) @ 7pm Jillian’s at Metreon 175 Fourth Street, San Francisco, CA At the corner of Howard and 4 th st.; only 2-min walk from Moscone Center

4. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Agenda What is a Data Dictionary? The MySQL Traditional Data Dictionary New Data Dictionary Benefits for users Q & A 1 2 3 4 5

5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Data Dictionary What is it

6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Data Dictionary Metadata is information about user data – User table structure – Column definitions – Index definitions – Foreign key definitions – Stored program definitions... Data Dictionary collects all metadata in RDBMS Definition

7. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | CREATE PROCEDURE p1(v INT) SQL SECURITY INVOKER BEGIN... END Data Dictionary Overview Data Dictionary Table DefinitionsSP DefinitionsView DefinitionsPluginsTime zonesPrivileges CREATE TABLE customers( id INT AUTO_INCREMENT... PRIMARY KEY (id), INDEX... FOREIGN KEY... )

8. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | MySQL Server Data Dictionary Definition 8 The ecosystem Query Executor Optimizer Performance Schema InnoDBSE SQL statement Client Parser Result Information Schema Data Dictionary

9. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | The MySQL Traditional Data Dictionary MySQL 5.6 and earlier

10. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | The MySQL Traditional Data Dictionary A mix of files and tables: – File based Tables: FRM Triggers: TRN, TRG... – Table based mysql.time_zone... InnoDB has a separate data dictionary

11. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | The MySQL Traditional Data Dictionary 11 Data Dictionary Files FRMTRGOPT System tables (mysql.  ) userproctime_zone InnoDB internal data dictionary CSV Archive InnoDB INFORMATION_SCHEMA File Scan Table Scan Intern. Access

12. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | The MySQL Traditional Data Dictionary Poor INFORMATION_SCHEMA performance Makes crash-safe / transactional DDL impossible Inconsistencies between files and tables Inconsistencies between DD in InnoDB and the server File-system dependency ( lower-case-table-names ) Makes replication of DDL statements difficult Too difficult to extend 12 Problems

13. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary A great leap forward

14. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Stored in InnoDB tables Reliable & crash-safe Single repository of metadata – for the MySQL server – for Storage Engines – for Plugins Redundancy Data Dictionary API INFORMATION_SCHEMA SQL VIEWs – Queries can be optimized – Improved performance Metadata versioning Extendable – Simplify metadata upgrades – Designed with plugins in mind New Data Dictionary : Main Features

15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary 15 Transition Data Dictionary Files FRMTRGOPT System tables (mysql.  ) userproctime_zone InnoDB internal dictionary CSV Archive InnoDB INFORMATION_SCHEMA DD Table SQL VIEW File Scan Table Scan Intern. Access

16. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary 16 Overview InnoDB Data Dictionary DD TableUser Table INFORMATION SCHEMA Views Archive User Table CSV User Table

17. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | InnoDB New Data Dictionary 17 Architecture Query Executor Parser Optimizer Data Dictionary Tablespace Data Dictionary Internal API Data Dictionary External API Plugin Archive User Table Tablespace Storage Engine

18. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary WL#6379: Schema definitions for new DD InnoDB Data Dictionary Tablespace Designed with INFORMATION_SCHEMA in mind Ability to store SE-specific data Use PK / FK to ensure consistency 18 Data Dictionary tables

19. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary API The only way to access Data Dictionary – For the server core – For Storage Engines – For plugins Hard to misuse Internal API (non-stable) and external API (stable) Provide a way to handle SE specific data 19 Design goals

20. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary SDI : Serialized Dictionary Information 20 Redundancy InnoDB Single User TS User Table SDI General TS User Table SDI System TS User Table SDI Data Dictionary User Table Definition Stored Program Privileges

21. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary 21 FRM shipping for MySQL Cluster? SELECT... FROM t1 FRM SDI

22. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | No FRM files New INFORMATION_SCHEMA Migrated to InnoDB: – time zone tables – help tables – mysql.plugins – mysql.servers Draft of Data Dictionary API http://labs.mysql.com – Do NOT use it in production – Install on a spare server MTR can be run 22 New Data Dictionary : Labs Release

23. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary Why does it matter for YOU?

24. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | INFORMATION_SCHEMA performance improvements Get per table size Blog post by Shlomi Noach: http://tinyurl.com/y8cnj7ohttp://tinyurl.com/y8cnj7o SELECT TABLE_SCHEMA, TABLE_NAME, ENGINE, SUM(DATA_LENGTH+INDEX_LENGTH) AS size, SUM(INDEX_LENGTH) AS index_size FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA NOT IN ('mysql', 'INFORMATION_SCHEMA') AND ENGINE IS NOT NULL GROUP BY TABLE_SCHEMA, TABLE_NAME; 24 VersionTime 5.7.5-m150.38 sec Labs Release0.08 sec

25. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Case: server crash There are some data files No / outdated backup FRM files lost How to use those data files? 25 The problem

26. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Traditional Data Dictionary “Move FRM files around” CREATE TABLE t1 (...) ALTER TABLE t1 DISCARD TABLESPACE ALTER TABLE t1 IMPORT TABLESPACE... Easy to make mistakes New Data Dictionary Self-descriptive tablespaces (SDI) Dedicated IMPORT statement Goal: error-proof procedure 26 Case: server crash

27. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | For Plugin Developers A way to access Data Dictionary Persistent Storage for plugins – Store/restore custom data Plugins can extend INFORMATION_SCHEMA & PERFORMANCE_SCHEMA – Add new tables – Add new columns to existing tables 27

28. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Data Dictionary Summary

29. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Data Dictionary : Takeaways Fundamental component in RDBMS Critical for performance Critical for reliability Critical for scalability 29

30. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | The MySQL Traditional Data Dictionary Mix of files and tables Server DD and InnoDB DD Inefficient INFORMATION_SCHEMA Difficult to extend New Data Dictionary Crash-safe InnoDB tables Single repository INFORMATION_SCHEMA as VIEWs Designed to be extendable Aims for backward compatibility Huge reengineering Data Dictionary : Takeaways

31. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Questions?

32. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | MySQL Community Reception @ Oracle OpenWorld Mingle with the MySQL community and the MySQL team from Oracle for a fun and informative evening! Time: September 30 (Tue) @ 7pm Jillian’s at Metreon 175 Fourth Street, San Francisco, CA At the corner of Howard and 4 th st.; only 2-min walk from Moscone Center

33. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle University MySQL Training Services Prepare Your Organization to Enable Reliable and High-Performance Web-Based Database Applications “Training and team skill have the most significant impact on overall performance of technology and success of technology projects.” - IDC, 2013 Premier Support customers eligible to save 20% on learning credits. Benefits  Expert-led training to support your MySQL learning needs  Flexibility to train in the classroom or online  Hands-on experience to gain real world experience  Key skills needed for database administrators and developers MySQL for Beginners  MySQL for Database Administrators  MySQL Performance Tuning  MySQL Cluster – NEW - Register Your Interest!  MySQL and PHP - Developing Dynamic Web Applications  MySQL for Developers  MySQL Developer Techniques  MySQL 5.6 Database Administrator  MySQL 5.6 Developer To find out more about available MySQL Training & Certification offerings, go to: education.oracle.com/mysqleducation.oracle.com/mysql Top Courses for Administrators and Developers Top Certifications RECENTLY RELEASED ALL NEW! MySQL Cluster Training To Register your interest to influence the schedule on this newly released course – go to education.oracle.com/mysql and click on the MySQL Cluster CourseMySQL Cluster Training

34. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Thank You! Copyright © 2014, Oracle and/or its affiliates. All rights reserved.