Features of TMySQL/TSpider port into MariaDB Tencent Game DBA Team

Agenda Tencent Game DBA Team TenDB and TenDB Cluster introduction What features have been port into MariaDB Roadmap of open source What will we do next Back to the open source

Tencent Game DBA Team Tencent Games DBA Team Member Hundreds of self-developed or delegated games，20000+ MySQL instances and 10000+ machines 50% of the Tencent’s revenue comes from games DBA Team Member 4 people work on MySQL/Spider source code development – mysql kernel developer 2 people work on DB automation management platform 24 people work on DB management and maintenance jobs

TenDB and TenDB Cluster Introduction TMySQL -> TenDB A MySQL branch maintained by Tencent Game DBA Team Based on MySQL 5.5.24/Percona 5.6.24(thread_pool feature) – large-scale use in production environment Based on MySQL 5.7 – in developing TSpider -> TenDB Cluster Base on Spider engine 3.1, started in late 2013 The largest Spider Cluster, 2000+instances Auto sharding and auto-online scale out TenDB hash(id)=0 hash(id)=1 hash(id)=2 hash(id)=.. Domain+Port Spider

Part1: What features have been ported into MariaDB TenDB features have been ported into MariaDB Version Core Features Description Release Time MariaDB Version TenDB 1.1 InnoDB Instant Add Column Add column finished in less than one second, no need to copy table 2012/12/10 MariaDB 10.3.2 TenDB 1.2 Memory optimization core bugs fixed Use tcmalloc to solve the memory fragmentation problem 5 important mysql bugs fixed 2013/3/20 TenDB 1.3 (In-Place) Upgrade enhance backup/import data Support MySQL5.0 —> TenDB inplace upgrade Parallel data import（A5 60% Z3 90% data import time recude） 2013/6/6 TenDB 1.4 SqlParse InnoDB Big Column Compression Integrated grammar, semantic check to OSS change subsystem, improve business change efficiency Storage-engine independent column compression enables blob/text/varbinary columns to be compressed 2014/10/7 TenDB 1.5 Binlog compression Read binlog speed limit Add new binlog events to store the compressed binlog and reduce the storage capacity restrict the speed at which the slave reads the binlog from the master 2015/06/07 MariaDB 10.2.3 TenDB 2.1 Parallel replication Extended binary log events, support for parallel replication and recovery based on table, more than 3 times more efficient 2016/1/1 TenDB 2.1.4 relay log compression 2017/9/7

Part2: What features have been ported into MariaDB TenDB Cluster features have been ported into MariaDB(immediately) Production Item Type Name Description Status MariaDB Version TenDB Cluster New Feature Connection Pool Allocate connections on demand and recycle them regularly, which can enhance the cluster performance has been Merged into Spider 3.3(2017.4.1) branch: bb-10.2-spider force index Push Down force index push down to the remote DB instances Optimization limit x,y optimization in most case, limit x,y will cost lot of overhead, the optimization calculate which exact shard holds the needed data pre_scan optimization The prefetch function (pre_scan) in Spider causes the net read timeout when a single shard is processed too long BUG Fixed Open table BUG solve a particular situation where cause infinite loops System Table BUG solve table’s name exceed 80 bytes cannot be created windows BUG solve the problem of debug on windows environment

Roadmap of open source 2017.10.9 MariaDB releases the alpha version of 10.3.2 Covering the two major features of TenDB: Instant Add Column and column compression 2017.04 The second time to participate the MariaDB Developer Conference Discuss the Instant Add Column design and implementation, 45 days integrated into MariaDB 10.3 Discuss the TenDB Cluster connection pool and other features and bugs fixed integrated to MariaDB 10.3, and Spider engine will be the first stable version of the release 2016.07 Invite Monty to Tencent Exchange Technology.We shared kernel development and online practice for TenDB and TenDB Cluster 2016.12 MariaDB release the Beta version of 10.2.3Covering the two features of TenDB Replication optimization:binlog compression and slave pull binlog speed limit 2016.10 Tencent Game DBA Team become bronze member of MariaDB Foundation Participate in the MariaDB Developer Conference for the first time Identified the two features of TenDB Replication optimization integrated into MariaDB 10.2.3 version 2016.08

What will we do next Make Spider engine more stable in MariaDB – work together with Kentoku Add distributed transaction support in TenDB Cluster Solve the transaction consistency in fault scenarios Push more features to MariaDB, we will benefit from this too Candidates: Parallel replication(table level) or Relay log compression? We are going to port the Flash Back feature(MariaDB) back to TenDB

Back to the open source Good people together will become more outstanding Open source process allows us to constantly improve Win-Win, Help us, help you Spend less time to migrate the code from our branch to the newest MySQL version More and more important features can be merged into MariaDB, which will influence many users, we can be proud to say that: this feature is from our team! We can get back from MariaDB Port improvements from MariaDB to TenDB Port some useful feature into TenDB