1. 1 Copyright 2008 Sun The World’s Most Popular Open Source Database Maria Engine Oscon, 2008-07-27 Michael Widenius MySQL Fellow

2. 2 Copyright 2008 Sun The World’s Most Popular Open Source Database Project Goals 1. To create an ACID and multi-version concurrency Control (MVCC) transactional storage engine that can function as the default non-transactional and the default transactional storage engine for MySQL. 2. To be a MyISAM replacement. This is possible because Maria can also be run in non-transactional mode and Maria supports same row formats and features as MyISAM 3. To create a Storage Engine for MySQL which is good for data warehousing (DW) purposes

3. 3 Copyright 2008 Sun The World’s Most Popular Open Source Database Why Falcon AND Maria? Maria and Falcon have architecture differences (feature/capability/performance -wise) ‏ – With Falcon+Maria we should be able to cover more scenarios and requirements that are important for MySQL users Maria to also target Data Warehousing - a growing market where MyISAM (the “core” of Maria) already has proven some success Maria can be used standalone outside of MySQL => Differences between the two engines are Good = Users have options to choose from based on their needs – Exact target market for engines will be more clear as the engine starts to be more deployed

4. 4 Copyright 2008 Sun The World’s Most Popular Open Source Database Project Team and allocation Michael “Monty” WideniusTechnical Lead Sanja ByelkinDeveloper Sergei GolubchikDeveloper Guilhem BichotDeveloper Jani TolonenQA We are going to to add 2-3 full time developers/QA persons to the team shortly

5. 5 Copyright 2008 Sun The World’s Most Popular Open Source Database Project Plan (as seen by Maria team) ‏ MySQL 5.1 + Maria release – Maria 1.0,”Crash Safe”, released in January 2008 – Maria 1.5, “Concurrent insert/select”, June 2008 Maria 1.5 is to be merged as part of formal MySQL 6.0 release MySQL 6.0 + Maria Release – "Maria 2.0 Transactional and ACID" alpha, by Q4/08 – "Maria 3.0 High Concurrency & Online Backup" alpha, by Q1/09 – "Maria 3.0 High Concurrency & Online Backup" GA, by Q2/09 MySQL 6.1 + Maria release – "Maria 4.0 Data Warehousing" alpha, by Q3/2009 Bitmap & XDB indexes Note: These are Maria releases, not official MySQL server releases!

6. 6 Copyright 2008 Sun The World’s Most Popular Open Source Database In-depth look into Maria What to expect of Maria 1.5 Technical goals How Maria 1.5 compares to MyISAM Resources for understanding more about Maria

7. 7 Copyright 2008 Sun The World’s Most Popular Open Source Database Technical goals of Maria 1/2 ● Multi-version concurrency Control (MVCC) and ACID ● MyISAM replacement (non transactional table support) ‏ ● Used for internal temporary tables in MySQL ● All indexes should have equal speed (clustered index is not on current road map). ● Allow 'any' length transactions to work (Having long running transactions will just cause more logs space to be used) ‏ ● Allow log shipping; that is, you can do incremental backups of Maria tables just by copying the Maria logs. ● Can be used as standalone library

8. 8 Copyright 2008 Sun The World’s Most Popular Open Source Database Technical goals of Maria 2/2 ● Allow copying of Maria tables between different Maria servers (under some well-defined constraints) ‏ ● Better blob handling (than that in MyISAM at least): ● No memory copying or extra memory used for blobs on insert/update. ● Blobs allocated in big sequential blocks - Less fragmentation over time ● Blobs are stored so that Maria can easily be extended to have access to any part of a blob with a single fetch in the future. ● Instant SELECT COUNT(*) and table checksums ● Efficient storage on disk (that is, low row data overhead, low page data overhead, and little lost space on pages). ● Small footprint, to make MySQL + Maria suitable for desktop and embedded applications.

9. 9 Copyright 2008 Sun The World’s Most Popular Open Source Database Maria 1.5 - Comparison to MyISAM 1/2 Maria 1.0 was basically a crash safe, non transactional version of MyISAM with a new cacheable row format (Released) Maria 1.5 is a crash safe, non transactional version of MyISAM with higher insert/select concurrency (Code complete). Note: The primary goal is to make Maria of good quality – Performance is not a primary goal for Maria 1.0-2.0, - to be addressed in Maria 3.0 after we know the basic works

10. 10 Copyright 2008 Sun The World’s Most Popular Open Source Database Advantages of Maria, Compared to MyISAM 1/2 Data, index and statistics are crash safe – On crash, things will rollback to state of the start of statement or last LOCK TABLES commands. Maria can replay everything from the log. – Including create/drop/rename/truncate tables – One can thus make a backup of Maria by just copying the log. Maria supports multiple concurrent insert's and select's on the same crash-safe table. LOAD INDEX can skip index blocks for not wanted indexes Maria has unit tests of most parts

11. 11 Copyright 2008 Sun The World’s Most Popular Open Source Database Advantages of Maria, Compared to MyISAM 2/2 Supports both crash safe (soon to be transactional) and not transactional tables. – Not transactional tables are not logged and rows uses less space (identical space & performance as MyISAM tables) ‏ – CREATE TABLE foo (...) TRANSACTIONAL=0|1 Supports all MyISAM row formats + PAGE format where data is stored in pages. PAGE format is: – The only crashsafe/transactional row format for Maria – Versioned, which allows you to have any number of concurrent insert & selects on the same tables (Maria 1.5) ‏ – Cached by page cache and should give a notable speed improvement on systems which bad data caching Safer data with table and (new) page checksums

12. 12 Copyright 2008 Sun The World’s Most Popular Open Source Database Differences between Maria and MyISAM Maria uses BIG (1G by default) log files Maria has a log control file (maria_log_control) and log files (maria_log.???????). – The log files can be automatically purged when not needed or purged on demand (after backup) ‏ Maria uses by default 8K pages (MyISAM 1K). – Maria should be faster on static size indexes but slower on variable length keys until we add a directory to index pages (Maria 2.0) ‏

13. 13 Copyright 2008 Sun The World’s Most Popular Open Source Database Disadvantages of Maria compared to MyISAM, that *will be fixed soon* Repair with many threads (maria_chk –parallel-recover) not yet implemented for Maria's PAGE format Maria doesn't support INSERT DELAYED (yet) ‏ Fulltext indexes are not yet crash safe Fulltext and GIS index are not yet versioned

14. 14 Copyright 2008 Sun The World’s Most Popular Open Source Database Differences to MyISAM that are not likely to be fixed No external locking – MyISAM has external locking, but is not much used Maria has one page size for both index and data (defined when Maria is used first time) ‏ – MyISAM supports different page sizes per index Maria has only one page cache Index number + checksums requires 5 extra byte per page Maria doesn't support MyISAM RAID (disabled in MyISAM too) ‏ Minimum data file size for BLOCK format is 16K (with 8K pages) ‏ Storage of very small rows (< 25 bytes) are not efficient for PAGE format. – Maria data pages in block format has an overhead of 10 byte/page and 5 byte/rows. Transaction and multiple concurrent writer support will use an extra overhead of 7 bytes for new rows, 14 bytes for deleted rows and 0 bytes for old compacted rows.

15. 15 Copyright 2008 Sun The World’s Most Popular Open Source Database Todo before beta release of 1.5 Test versioning of indexes (Data and statistics are already versioned and tested) ‏ Parallel-recovery (rolling feature) ‏ Fixing a couple of (minor) reported bugs Should happen in August

16. 16 Copyright 2008 Sun The World’s Most Popular Open Source Database Row format Transactio n manager Page cache MySQL Control file The handler: class ha_maria ISAM read row write/update/del ete B-tree indexes search/next/prev /... Recovery Checkpoi nts Log handler Fixed Dynamic Blocks create drop check repair...

17. 17 Copyright 2008 Sun The World’s Most Popular Open Source Database New rowformat Record s Bitmap; 3 bits per page Head page; Start page for rows; Up to 255 rows per page Blob data Tail page; End of blob/long row data; Up to 255 tails per page 256 Meg Bitmap; 3 bits per page

18. 18 Copyright 2008 Sun The World’s Most Popular Open Source Database XDB indexes (for Maria 4.0) ‏ Record s 1000 minmax... minmax fields

19. 19 Copyright 2008 Sun The World’s Most Popular Open Source Database Maria - resources More information/Questions/Feedback: Maria roadmap, algorithms and internals are described in detail at http://forge.mysql.com/worklog/task.php?id=3871http://forge.mysql.com/worklog/task.php?id=3871 End user documentation about Maria can be found at http://dev.mysql.com/doc/refman/5.1/en/storage-engines.htmlhttp://dev.mysql.com/doc/refman/5.1/en/storage-engines.html Monty's blog at: http://monty-says.blogspot.com/ will have all important updates to the Maria project.http://monty-says.blogspot.com/ You can follow and participate in all Maria development discussions: maria@lists.mysql.com or at the Maria forum at: http://forums.mysql.com, section Mariamaria@lists.mysql.com You can report bugs and check bugs in Maria in the MySQL bugs system at http://bugs.mysql.com/