1. © Continuent 2010 Liberating Your Data From MySQL: Cross-Database Replication to the Rescue! Robert Hodges and Linas Virbalas Continuent, Inc.

2. © Continuent 2010 Agenda /Introductions /What is Tungsten? /A Word About MySQL Replication /Tungsten’s MySQL to PostgreSQL Replication /Demo! /Questions and Answers

3. © Continuent 2010 Introductions

4. About Continuent /Our Value: Make open source as robust as commercial databases Enterprise capability Cloud flexibility Open source cost /Our Solution: Tungsten Database Clustering /Our Company: Founded in Finland, HQ in USA Venture-backed US/European presence Deep experience in databases and distributed systems

5. © Continuent 2010 What is Tungsten and How Does It Work?

6. © Continuent 2010 What Is Tungsten? /Tungsten creates data services using off- the-shelf open source databases to: 1.Ensure reliable, fast access to data 2.Simplify common management tasks 3.Enable disaster recovery …Without changing applications

7. © Continuent 2010 Apache/Mod_PHP libmysqlclient.a Connector Tungsten 1.3 Data Service Architecture Apache/Mod_PHP DBMS Replicator Manager DBMS Replicator Manager DBMS Replicator Manager libmysqlclient.a Master Slave Slave Connector

8. © Continuent 2010 Technology: Replication Pipelines THL SlaveDBMS ApplierExtractorFiltersApplierExtractorFilters StageStage Pipeline Tungsten Replicator Process Transaction History Log Master THL (via network)

9. © Continuent 2010 So Why Are We Here? Tungsten can replicate in real time from MySQL to PostgreSQL

10. © Continuent 2010 Why Would You Want to Do That? /Feed MySQL merchant transactions to PostgreSQL applications /Feed PG-based data warehouse and reporting /Migrate MySQL data to PG with minimum downtime /Plus Scott McNealy says it’s good to share MySQL App 1 MySQL App 2 MySQL App 3 PG App

11. © Continuent 2010 A Word About MySQL Replication

12. © Continuent 2010 Writeable slaves Slave readableSlave writeable MySQL Statement Based++ MySQL Row Based++ MySQL Mixed++ PostgreSQL WAL Shipping-- PostgreSQL Streaming Replication (Hot Standby) +- /Tungsten for MySQL You can both read and write to the DB operating in slave mode This, potentially, allows you to break consistency In some scenarios that is a plus /Tungsten for PostgreSQL In Streaming Replication you can read from slaves Still, you cannot write to them In effect, you cannot have different data/structure on a slave

13. © Continuent 2010 Statement and Row Based Replication /Statement Based Replication /Row Based Replication CREATE TABLE t (id INT, name VARCHAR(12)); ALTER TABLE t ADD COLUMN id2 INT; INSERT INTO t VALUES (1, ‘Sun’, 999); UPDATE t SET name = ‘Moon’; CREATE TABLE t (id INT, name VARCHAR(12)); ALTER TABLE t ADD COLUMN id2 INT; { INSERT, t, (1, ‘Sun’, 999) }; { UPDATE, t, id == 1, (NULL, ‘Moon’, NULL) };

14. © Continuent 2010 Logical vs. Physical Replication LogicalPhysical MySQL Statement Basedx MySQL Row Basedx MySQL Mixedx PostgreSQL WAL Shippingx PostgreSQL Streaming Replicationx Filters (data transformation) possible+- Different data/structure on slave possible +-

15. © Continuent 2010 Tungsten MySQL -> PostgreSQL Replication

16. © Continuent 2010 MySQL -> PostgreSQL: Who’s Who? BinLogs Master DB (Binlogs enabled) Slave DB MySQL setup to run as MySQL master PostgreSQL setup to accept writes – Continuous Recovery is OFF

17. © Continuent 2010 MySQL -> PostgreSQL: Tungsten Replicator Doing the Job BinLogs Tungsten Replicator Platform- Independent SQL Events Log Record Master DB (Binlogs enabled) Slave DB JDBC SQL Requests

18. © Continuent 2010 MySQL -> PostgreSQL: Tungsten Replicator Components BinLogs Master Replicator MySQL Extractor Transaction History Log Slave Replicator PostgreSQL Applier Transaction History Log Platform- Independent SQL Events Log Record Master DB (Binlogs enabled) Slave DB JDBC SQL Requests Filters

19. © Continuent 2010 MySQL -> PostgreSQL: Challenges /Data Type Differences /Default (Implicitly Defined) Schema Selection /SQL Dialect Differences Statement Replication vs. Row Replication /Character Sets and Binary Data

20. © Continuent 2010 MySQL -> PostgreSQL: Data Types MySQLPostgreSQL !TINYINTSMALLINT INTEGER BIGINT !CHAR(1)CHAR(5) = {‘true’, ‘false’} CHAR(x) VARCHAR(x) DATE TIMESTAMP !TEXT (diff. sizes)TEXT !BLOBBYTEA … / /Note the type differences between MySQL and PG

21. © Continuent 2010 MySQL -> PostgreSQL: Default Schema MySQL ImplicitMySQL Explicit CREATE SCHEMA s; USE s; !CREATE TABLE t (i int);CREATE TABLE s.t (i int); !INSERT INTO t (1);INSERT INTO s.t (1); / /MySQL: Trivial to use `USE` / /MySQL: Going without `USE` generates different events / /PG: Extract the default schema from the event / /PG: Set it before applying MySQLPostgreSQL USE s;>SET search_path TO s, "$user”;

22. © Continuent 2010 MySQL -> PostgreSQL: SQL Dialect MySQLPostgreSQL CREATE TABLE complex (id INTEGER AUTO_INCREMENT PRIMARY KEY, i INT); CREATE TABLE complex (id SERIAL PRIMARY KEY, i INT); CREATE TABLE dt (i TINYINT); CREATE TABLE dt (i SMALLINT); … / /Differences between DDL and DML statement SQL dialects / /Row Replication resolves issues rising from differences in DML, but still leaves DDL to handle / /Tungsten Replicator Filters come to the rescue! Simple to develop Java or JavaScript extensions Event structure IN -> Filter -> Event structure OUT

23. © Continuent 2010 MySQL -> PostgreSQL: Charsets MySQLPostgreSQL INSERT INTO embedded_blob (key, data) VALUES (1, ‘ ?\0^Es\0^\0\’’) ARGH!!! (SQL statement fails) create table xlate(id int, d1 varchar(25) character set latin1, d2 varchar(25) character set utf8); ARGH!!! (no way to translate to common charset) / /Statement replication: MySQL syntax is “permissive” / /Embedded binary / alternate charsets / /Different charsets for different clients / /Row replication: database/table/column charsets may differ / /Answer: Stick with one character set throughout; use row replication if you need to move binary data

24. © Continuent 2010 MySQL-> PostgreSQL: What’s Left? /We have covered the basics but there is more… /Initial provisioning from MySQL to PostgreSQL /Data transformation /Making it go *really* fast (think parallel replication)

25. © Continuent 2010 DEMO

26. Q & A

27. © Continuent 2010 HQ and Americas 560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel (866) 998-3642 Fax (408) 668-1009 e-mail: sales@continuent.com EMEA and APAC Lars Sonckin kaari 16 02600 Espoo, Finland Tel +358 50 517 9059 Fax +358 9 863 0060 Contact Information Continuent Web Site: http://www.continuent.com