1. Features of TMySQL/TSpider port into MariaDB
Tencent Game DBA Team

2. Agenda Tencent Game DBA Team TenDB and TenDB Cluster introduction
What features have been port into MariaDB
Roadmap of open source
What will we do next
Back to the open source

3. Tencent Game DBA Team Tencent Games DBA Team Member
Hundreds of self-developed or delegated games， MySQL instances and machines
50% of the Tencent’s revenue comes from games
DBA Team Member
4 people work on MySQL/Spider source code development – mysql kernel developer
2 people work on DB automation management platform
24 people work on DB management and maintenance jobs

4. TenDB and TenDB Cluster Introduction
TMySQL -> TenDB
A MySQL branch maintained by Tencent Game DBA Team
Based on MySQL /Percona (thread_pool feature) – large-scale use in production environment
Based on MySQL 5.7 – in developing
TSpider -> TenDB Cluster
Base on Spider engine 3.1, started in late 2013
The largest Spider Cluster, 2000+instances
Auto sharding and auto-online scale out
TenDB
hash(id)=0
hash(id)=1
hash(id)=2
hash(id)=..
Domain+Port
Spider

5. Part1: What features have been ported into MariaDB
TenDB features have been ported into MariaDB
Version
Core Features
Description
Release Time
MariaDB Version
TenDB 1.1
InnoDB Instant Add Column
Add column finished in less than one second, no need to copy table
2012/12/10
MariaDB
TenDB 1.2
Memory optimization
core bugs fixed
Use tcmalloc to solve the memory fragmentation problem
5 important mysql bugs fixed
2013/3/20
TenDB 1.3
(In-Place) Upgrade
enhance backup/import data
Support MySQL5.0 —> TenDB inplace upgrade
Parallel data import（A5 60% Z3 90% data import time recude）
2013/6/6
TenDB 1.4
SqlParse
InnoDB Big Column Compression
Integrated grammar, semantic check to OSS change subsystem, improve business change efficiency
Storage-engine independent column compression enables blob/text/varbinary columns to be compressed
2014/10/7
TenDB 1.5
Binlog compression
Read binlog speed limit
Add new binlog events to store the compressed binlog and reduce the storage capacity restrict the speed at which the slave reads the binlog from the master
2015/06/07
MariaDB
TenDB 2.1
Parallel replication
Extended binary log events, support for parallel replication and recovery based on table, more than 3 times more efficient
2016/1/1
TenDB 2.1.4
relay log compression
2017/9/7

6. Part2: What features have been ported into MariaDB
TenDB Cluster features have been ported into MariaDB(immediately)
Production
Item Type
Name
Description
Status
MariaDB Version
TenDB Cluster
New Feature
Connection Pool
Allocate connections on demand and recycle them regularly, which can enhance the cluster performance
has been Merged into Spider 3.3( )
branch: bb-10.2-spider
force index Push Down
force index push down to the remote DB instances
Optimization
limit x,y optimization
in most case, limit x,y will cost lot of overhead, the optimization calculate which exact shard holds the needed data
pre_scan optimization
The prefetch function (pre_scan) in Spider causes the net read timeout when a single shard is processed too long
BUG Fixed
Open table BUG
solve a particular situation where cause infinite loops
System Table BUG
solve table’s name exceed 80 bytes cannot be created
windows BUG
solve the problem of debug on windows environment

7. Roadmap of open source
MariaDB releases the alpha version of Covering the two major features of TenDB: Instant Add Column and column compression
The second time to participate the MariaDB Developer Conference
Discuss the Instant Add Column design and implementation, 45 days integrated into MariaDB 10.3
Discuss the TenDB Cluster connection pool and other features and bugs fixed integrated to MariaDB 10.3, and Spider engine will be the first stable version of the release
Invite Monty to Tencent Exchange Technology.We shared kernel development and online practice for TenDB and TenDB Cluster
MariaDB release the Beta version of Covering the two features of TenDB Replication optimization:binlog compression and slave pull binlog speed limit
Tencent Game DBA Team become bronze member of
MariaDB Foundation
Participate in the MariaDB Developer Conference for the first time Identified the two features of TenDB Replication optimization integrated into MariaDB version

8. What will we do next
Make Spider engine more stable in MariaDB – work together with Kentoku
Add distributed transaction support in TenDB Cluster
Solve the transaction consistency in fault scenarios
Push more features to MariaDB, we will benefit from this too
Candidates: Parallel replication(table level) or Relay log compression?
We are going to port the Flash Back feature(MariaDB) back to TenDB

9. Back to the open source
Good people together will become more outstanding
Open source process allows us to constantly improve
Win-Win, Help us, help you
Spend less time to migrate the code from our branch to the newest MySQL version
More and more important features can be merged into MariaDB, which will influence many users, we can be proud to say that: this feature is from our team!
We can get back from MariaDB
Port improvements from MariaDB to TenDB
Port some useful feature into TenDB