1. Instant Add Columns in MySQL
Vin Chen
Tencent Game

2. Who I am Tencent Game DBA Team Profiles Contributor to Oracle MySQL
Our Team has been serving game business for a few years. The mission of the DBA Team is to provide stable and efficient online storage services for Tencent Games. As the storage requirements from game business evolve, we keep extending features of our database and improving its performance. We spared no effort to provide reliable, scalable and performant database solutions.
Profiles
Team leader of Tencent DBA Developer Team
Senior engineer of Tencent
More than 7 years MySQL development
Focus on open source storage solutions(MySQL, MongoDB, Redis, etc.)
Contributor to Oracle MySQL
Author of instant add columns

3. Background Why we need instant add columns
High Frequency of Adding columns to BIG Table in Production Env
Expensive operation with Adding columns to BIG table
Causing downtime in the Game business
“Instant add columns” in Tencent Game
First released in July 2012, TenDB
Based on mysql , and ,
Contribution to Oracle MySQL in 2017
Merged and RELEASED in dated July 2018

4. Implementation
Row Format
Dictionary
DDL & DML
Redo

5. Non-NULL Variable-Length Array
Row Format
Record Format Characteristic (Compact/Dynamic)
Storage order is the same as definition order (except primary key)
NULL field not stored. A bitmap is used to indicate which fields are NULL.
Field number of a record is index->n_fields
Non-NULL Variable-Length Array
Nullable Bitmap
Record Extra Bytes
Record Header
Primary Key
Transaction ID
Roll Ptr
Other not null Fields
Record Ptr
n_owned+Info Bits
(4Bits + 4Bits, 1 Byte)
Status + Heap No
(3Bits + 13Bits, 2 Bytes)
Next Ptr
(2 Bytes)

6. Row Format Record Format Changes(Compact/Dynamic)
Field Count: the field count of the record(≤ index.n_fields)
Forward compatibility
Info Bits(higher 4 bits, 2 reserved bits)
REC_INFO_MIN_REC_FLAG 0x10
REC_INFO_DELETED_FLAG 0x20
REC_INFO_INSTANT_FLAG 0x80
Only leaf pages of cluster index are affected
Record Header
Primary Key
Transaction ID
Roll Ptr
Other not null Fields
/* The 0x40UL can also be used in the future */
/* The instant ADD COLUMN flag. When it is set to 1, it means this record
was inserted/updated after an instant ADD COLUMN. */
Record Ptr
Non-NULL Variable-Length Array
Nullable Bitmap
Record Extra Bytes
Non-NULL Variable-Length Array
Nullable Bitmap
Field Count
Record Extra Bytes
n_owned+Info Bits
(4Bits + 4Bits, 1 Byte)
Status + Heap No
(3Bits + 13Bits, 2 Bytes)
Next Ptr
(2 Bytes)

7. Row Format Record Format Changes(Compact/Dynamic)
If REC_INFO_INSTANT_FLAG is set, store the “Field count” using a variable-length encoding of 2 bytes
REC_INFO_INSTANT_FLAG would be set for insert/update after instant adding columns.
If REC_INFO_INSTANT_FLAG is zero, there are two possibilities:
Instant adding columns have never happened in this table.
The record is inserted or updated before the first time of instant adding columns.
n_core_fields: The field count before the first time instant adding columns happened.

8. Dictionary New metadata Store n_core_fields Store default value
When the first time instant adding columns happens, we need to store the original field count of a table
Update sys_tables set (mix_len = mix_len | n_core_fields << 16) where table = :table_id;
Store default value
A new system table sys_columns_added
Dictionary loading
When loading a table from dictionary
n_core_fields = sys_table.mix_len >> 16
cluster_index.n_core_fields = n_core_fields > 0 ? n_core_fields : cluster_index.n_fields.
dict_load_sys_columns_added() after dict_load_sys_columns()

9. Dictionary Examples MySQL 8 TenDB Use test;
create table t1(id int primary key, c1 int);
insert into t1 values(1,1);
alter table t1 add column d1 varchar(20) not null default 'abc‘, algorithm=‘instant’;
alter table t1 alter column d1 set default 'bcd';
Examples
MySQL 8
TenDB
The number of columns when the table undergoes first instant ADD COLUMN and all default value of newly added columns are stored in the data dictionary. These two pieces of information are stored in se_private_data column of the data dictionary tables.

10. DDL Key Logic Create table: no behavior change Alter table:
Instant adding column
No need to copy data
Insert or change the related metadata
Other: no behavior change
Drop/Truncate table:
Delete the related metadata
Update sys_tables set (mix_len = mix_len | n_core_fields << 16) where table = :table_id; (First time)
Insert into sys_columns values(…);
Insert into sys_columns_added values(:table_id, :pos, <default value>);

11. DML Key Logic insert: delete: update:
Before the first time Instant adding columns happened, no behavior change
Otherwise, REC_INFO_INSTANT_FLAG would be set.
delete:
No behavior change, just mark the delete flag;
update:
For updating old version record, if the added columns changed or the length of column changed, it will use non-inplace update(delete + insert)
Otherwise, no behavior change.

12. DML
Key Logic
select:

13. Redo Redo a mix of physical and logical operations
n_core_fields need to be stored in dict object of redo log
default values is useless for recovery

14. Summary Advantage Limitations Change metadata only, very fast
No need to copy data
No need to take twice the space
Forward compatibility with MySQL
Limitations
Only support adding columns in one statement, that is if there are other non-INSTANT operations in the same statement, it can’t be done instantly
Only support adding columns at last, not in the middle of existing columns
Not support COMPRESSED row format, which is seldom used
Not support a table which already has any fulltext index
Not support any table residing in DD tablespace
Not support temporary table(it goes with COPY)

15. Mass production environment deployment
MySQL Usage In Tencent Games
Load Test
Functional Test
Grayscale deployment
Mass production environment deployment
400+ Games
10,000+ Machines
20,000+ Instances
20+ FailOver Per Day 17
Table DDL Per week
All instances use TenDB, which base on Oracle MySQL 5.5/5.6/5.7
Our Plan of Deploying MySQL 8.0

16. Thank you for your patience!