© Continuent 2010 Liberating Your Data From MySQL: Cross-Database Replication to the Rescue! Robert Hodges and Linas Virbalas Continuent, Inc.

© Continuent 2010 Agenda /Introductions /What is Tungsten? /A Word About MySQL Replication /Tungsten’s MySQL to PostgreSQL Replication /Demo! /Questions and Answers

© Continuent 2010 Introductions

About Continuent /Our Value: Make open source as robust as commercial databases Enterprise capability Cloud flexibility Open source cost /Our Solution: Tungsten Database Clustering /Our Company: Founded in Finland, HQ in USA Venture-backed US/European presence Deep experience in databases and distributed systems

© Continuent 2010 What is Tungsten and How Does It Work?

© Continuent 2010 What Is Tungsten? /Tungsten creates data services using off- the-shelf open source databases to: 1.Ensure reliable, fast access to data 2.Simplify common management tasks 3.Enable disaster recovery …Without changing applications

© Continuent 2010 Apache/Mod_PHP libmysqlclient.a Connector Tungsten 1.3 Data Service Architecture Apache/Mod_PHP DBMS Replicator Manager DBMS Replicator Manager DBMS Replicator Manager libmysqlclient.a Master Slave Slave Connector

© Continuent 2010 Technology: Replication Pipelines THL SlaveDBMS ApplierExtractorFiltersApplierExtractorFilters StageStage Pipeline Tungsten Replicator Process Transaction History Log Master THL (via network)

© Continuent 2010 So Why Are We Here? Tungsten can replicate in real time from MySQL to PostgreSQL

© Continuent 2010 Why Would You Want to Do That? /Feed MySQL merchant transactions to PostgreSQL applications /Feed PG-based data warehouse and reporting /Migrate MySQL data to PG with minimum downtime /Plus Scott McNealy says it’s good to share MySQL App 1 MySQL App 2 MySQL App 3 PG App

© Continuent 2010 A Word About MySQL Replication

© Continuent 2010 Writeable slaves Slave readableSlave writeable MySQL Statement Based++ MySQL Row Based++ MySQL Mixed++ PostgreSQL WAL Shipping-- PostgreSQL Streaming Replication (Hot Standby) +- /Tungsten for MySQL You can both read and write to the DB operating in slave mode This, potentially, allows you to break consistency In some scenarios that is a plus /Tungsten for PostgreSQL In Streaming Replication you can read from slaves Still, you cannot write to them In effect, you cannot have different data/structure on a slave

© Continuent 2010 Statement and Row Based Replication /Statement Based Replication /Row Based Replication CREATE TABLE t (id INT, name VARCHAR(12)); ALTER TABLE t ADD COLUMN id2 INT; INSERT INTO t VALUES (1, ‘Sun’, 999); UPDATE t SET name = ‘Moon’; CREATE TABLE t (id INT, name VARCHAR(12)); ALTER TABLE t ADD COLUMN id2 INT; { INSERT, t, (1, ‘Sun’, 999) }; { UPDATE, t, id == 1, (NULL, ‘Moon’, NULL) };

© Continuent 2010 Logical vs. Physical Replication LogicalPhysical MySQL Statement Basedx MySQL Row Basedx MySQL Mixedx PostgreSQL WAL Shippingx PostgreSQL Streaming Replicationx Filters (data transformation) possible+- Different data/structure on slave possible +-

© Continuent 2010 Tungsten MySQL -> PostgreSQL Replication

© Continuent 2010 MySQL -> PostgreSQL: Who’s Who? BinLogs Master DB (Binlogs enabled) Slave DB MySQL setup to run as MySQL master PostgreSQL setup to accept writes – Continuous Recovery is OFF

© Continuent 2010 MySQL -> PostgreSQL: Tungsten Replicator Doing the Job BinLogs Tungsten Replicator Platform- Independent SQL Events Log Record Master DB (Binlogs enabled) Slave DB JDBC SQL Requests

© Continuent 2010 MySQL -> PostgreSQL: Tungsten Replicator Components BinLogs Master Replicator MySQL Extractor Transaction History Log Slave Replicator PostgreSQL Applier Transaction History Log Platform- Independent SQL Events Log Record Master DB (Binlogs enabled) Slave DB JDBC SQL Requests Filters

© Continuent 2010 MySQL -> PostgreSQL: Challenges /Data Type Differences /Default (Implicitly Defined) Schema Selection /SQL Dialect Differences Statement Replication vs. Row Replication /Character Sets and Binary Data

© Continuent 2010 MySQL -> PostgreSQL: Data Types MySQLPostgreSQL !TINYINTSMALLINT INTEGER BIGINT !CHAR(1)CHAR(5) = {‘true’, ‘false’} CHAR(x) VARCHAR(x) DATE TIMESTAMP !TEXT (diff. sizes)TEXT !BLOBBYTEA … / /Note the type differences between MySQL and PG

© Continuent 2010 MySQL -> PostgreSQL: Default Schema MySQL ImplicitMySQL Explicit CREATE SCHEMA s; USE s; !CREATE TABLE t (i int);CREATE TABLE s.t (i int); !INSERT INTO t (1);INSERT INTO s.t (1); / /MySQL: Trivial to use `USE` / /MySQL: Going without `USE` generates different events / /PG: Extract the default schema from the event / /PG: Set it before applying MySQLPostgreSQL USE s;>SET search_path TO s, "$user”;

© Continuent 2010 MySQL -> PostgreSQL: SQL Dialect MySQLPostgreSQL CREATE TABLE complex (id INTEGER AUTO_INCREMENT PRIMARY KEY, i INT); CREATE TABLE complex (id SERIAL PRIMARY KEY, i INT); CREATE TABLE dt (i TINYINT); CREATE TABLE dt (i SMALLINT); … / /Differences between DDL and DML statement SQL dialects / /Row Replication resolves issues rising from differences in DML, but still leaves DDL to handle / /Tungsten Replicator Filters come to the rescue! Simple to develop Java or JavaScript extensions Event structure IN -> Filter -> Event structure OUT

© Continuent 2010 MySQL -> PostgreSQL: Charsets MySQLPostgreSQL INSERT INTO embedded_blob (key, data) VALUES (1, ‘ ?\0^Es\0^\0\’’) ARGH!!! (SQL statement fails) create table xlate(id int, d1 varchar(25) character set latin1, d2 varchar(25) character set utf8); ARGH!!! (no way to translate to common charset) / /Statement replication: MySQL syntax is “permissive” / /Embedded binary / alternate charsets / /Different charsets for different clients / /Row replication: database/table/column charsets may differ / /Answer: Stick with one character set throughout; use row replication if you need to move binary data

© Continuent 2010 MySQL-> PostgreSQL: What’s Left? /We have covered the basics but there is more… /Initial provisioning from MySQL to PostgreSQL /Data transformation /Making it go *really* fast (think parallel replication)

© Continuent 2010 DEMO

Q & A

© Continuent 2010 HQ and Americas 560 S. Winchester Blvd., Suite 500 San Jose, CA Tel (866) Fax (408) EMEA and APAC Lars Sonckin kaari Espoo, Finland Tel Fax Contact Information Continuent Web Site: