1. Transaction-based Grid Data Replication Using OGSA-DAI Presented by Yin Chen February 2007

2. Initial copying of data & synchronization of updating Is not Cashing –Client phenomenon –Only for improving response time Is not a Backup –Not automatically overwritten when the original data is modified –Normally, cannot directly access What is replication? What is replication?

3. Data consolidation (central audit & analyse) Data distribution (for branch labs) Performance –Access efficiency (moving data near processing) –Load balancing (distributing access load) –Security (data protection) –Availability (off-line access) –Reliability (disaster recovery, avoiding single point of failure) Why do we need it?

4. How to copy the large data among heterogeneous DBs How to maintain the consistency of data in a highly distributed network environment How to discover & self-repair the dead parts Challenges of Grid database replication

5. Existing Grid “replication” systems  E.g. the EDG replica manager/ the Globus data replication service/ SRB  Support large dataset copying  Yet, merely deal with files  Too simple (e.g. not support updating, database replication, etc.)  Not consistent Relational database replication tools  E.g. Oracle/ Sybase/ DB2/ MySQL replication  Very flexible (e.g., portion copy, bi-direction update)  Yet, not suit for virtual organizations (e.g. can’t copy large data/ difficult to search for replicas) Problems of existing technologies

6. Architecture Metadata Catalogue Relational Database Replication Mechanism Replication Control Service Transfer Service Data Resource Data Replica Data flow directions

7. Request Replication Control Service Metadata Search Engine Metadata Register Initiator Selector Starter Metadata Catalogue Relational Database Replication Mechanism Transfer Service Data Resource Replication Target Replication control workflow

8. OGSA-DAI activities (ongoing) High-level APIs to interact with relational replication mechanisms: 3 CreateReplicaDatabase() 3 DropReplicaDatabase() 3 ConfigReplication() 3 CleanUp() -- to clean up replication configuration 3 StartReplication() 3 StopReplication () 3 MonitorReplication() -- to check the status of each process Control the workflow of data replication, i.e. sequence.addChild(createDB2RelicaDB); sequence.addChild(configDB2Replication); sequence.addChild(startDB2Replication);

9. IBM DB2 SQL Replication IBM Replication Admin: create replication criteria  control table Capture: use log/trigger to capture the changes  temp table Apply: scheduled apply transactions accumulated  target DB Alert Monitor: monitor and notify users Supports: after-image copy / before-image copy (can rollback) Allows subset/simple view/ complex joins & unions copy Asynchronous replication, allows specifying schedule

10. Features Combine Relational Database Replication with Grid technologies, to gain benefits from both 3 Keep the features of relational database replication 3 Supporting more scalable, secure, high performance data access Explore the abilities of OGSA-DAI to control workflows

11. Information Project members: Dave Berry (NeSC, UK) Patrick Dantressangle (IBM, Hursley) Yin Chen (NeSC, UK) Simon Laws (IBM, Hursley) Project website: http://www.aiai.ed.ac.uk/~ychen/ibm_ogsadai/ibm-ogsadai-index.html