Architecture and Interface of Scalable Distributed Database System SD-SQL Server Soror SAHRI Witold LITWIN Thomas SCHWARTZ Soror.sahri@dauphine.fr Witold.litwin@dauphine.fr tjschwarz@scu.edu Ceria Laboratory Comp. Eng. Dep. Paris-Dauphine University Santa Clara U. The IASTED International Conference on DBA 2006 IASTED - DBA 2006, Innsbruck, February 14-16, 2006

IASTED-DBA 2006, Innsbruck, Feb 15th, 2006 Outline Introduction Overall Architecture Application Interface Implementation Conclusion IASTED-DBA 2006, Innsbruck, Feb 15th, 2006 IASTED - DBA 2006, Innsbruck, February 14-16, 2006

Introduction Most of DBSs have distributed/parallel versions SQL Server, Oracle, DB2 DBSs do not provide dynamically scalable tables. All require manual repartitioning when tables scale-up. Solution? A Scalable Distributed Database System: SD-DBS IASTED-DBA 2006, Innsbruck, Feb 15th, 2006

IASTED-DBA 2006, Innsbruck, Feb 15th, 2006 Introduction Applies SDDS technology to DBSs Why SDDSs? Provide many scalable distributed partitioning schemes. LH*, RP*, k-RP*, LH*RS… These schemes can serve as the basis for SD-DBS architecture IASTED-DBA 2006, Innsbruck, Feb 15th, 2006

SD-SQL Server Architecture User/Application User/Application sd_select sd_insert SD-SQL Server Managers SD-SQL client SD-SQL server SD-SQL server SD-SQL peer Linked SQL Servers D1_T T _D1_T _D1_T Split _D1_T NDBs D1 D2 Di Di+1 IASTED-DBA 2006, Innsbruck, Feb 15th, 2006

SD-SQL Server Architecture Nodes, SDBs, NDBs MDB Node1 Node2 Node3 Node i …… DB1 DB2 DB1 DB2 DB1 DB2 SDB DB1 SDB IASTED-DBA 2006, Innsbruck, Feb 15th, 2006

A scalable (distributed) table is a collection of segments SD-SQL Server Architecture Scalable Tables: Segments A scalable (distributed) table is a collection of segments Segments are SQL tables A scalable table has, initially, only one primary segment At some server or peer NDB The number of segments in a scalable table is variable. If a segment overflows, its split is triggered IASTED-DBA 2006, Innsbruck, Feb 15th, 2006 IASTED - DBA 2006, Innsbruck, February 14-16, 2006

SD-SQL Server Architecture Scalable Tables: Split A split occurs when an insert overflows the segment capacity Splits produce other segments for a scalable table. Each is located at a different NDB Within the SDB If there is not enough NDBs, splits dynamically append new ones IASTED-DBA 2006, Innsbruck, Feb 15th, 2006

SD-SQL Server Architecture Images Images hide the scalable table segments An image is a distributed updateable partitioned view of a scalable table Union-all view with check constraints An image presents the scalable table partitioning It do not address any new segments resulted from a split Each scalable table has only one primary image and one or several secondary images IASTED-DBA 2006, Innsbruck, Feb 15th, 2006

… SD-SQL Server Architecture Images T Scalable Table Primary Image DB1 SDB N1.DB1 N2.DB1 Ni.DB1 T Scalable Table … Primary Image CREATE VIEW T AS SELECT * FROM N1.DB1._N3_T UNION ALL SELECT * FROM N2.DB1._N3_T UNION ALL SELECT * FROM Ni.DB1._N3_T IASTED-DBA 2006, Innsbruck, Feb 15th, 2006

SD-SQL Server Application Interface The application interface manipulates scalable tables through SD-SQL Server commands. The SD-SQL Server commands start with ‘sd_’ to distinguish from SQL Server commands for static tables. INSERT sd_insert CREATE TABLE sd_create_table IASTED-DBA 2006, Innsbruck, Feb 15th, 2006 IASTED - DBA 2006, Innsbruck, February 14-16, 2006

SD-SQL Server Application Interface Use of the SkyServer DB as benchmark http://research.microsoft.com/~gray/SDSS PhotoObj table as a scalable table. PhotoObj has 158,426 tuples (about 260 MB) Use of the laboratory machines Ceria, Dell1, Dell2…. IASTED-DBA 2006, Innsbruck, Feb 15th, 2006

SD-SQL Server Application Interface Nodes Management Node Creation sd_create_node ‘Dell1’ /* Server by default */ sd_create_node ‘Ceria’, ‘client’ Node Alteration sd_alter_node ‘Ceria’, ‘ADD server’ /* Becomes peer*/ Node Removal sd_drop_node ‘Ceria’ IASTED-DBA 2006, Innsbruck, Feb 15th, 2006 IASTED - DBA 2006, Innsbruck, February 14-16, 2006

SD-SQL Server Application Interface SDB/NDB Management SDB Creation sd_create_scalable_database ‘SkyServer’, ‘Dell1’, ‘Server’, 2 /* Creates the primary SkyServer NDB as well at Dell1*/ SDB Alteration sd_create_node_database ‘SkyServer’, ‘Ceria’, ‘Client’ SDB Removal sd_drop_scalable_database ‘SkyServer’ IASTED-DBA 2006, Innsbruck, Feb 15th, 2006

SD-SQL Server Application Interface Scalable Tables Scalable Table Creation sd_create_table ‘PhotoObj (objid BIGINT PRIMARY KEY…)’, 10000 Scalable Table Alteration sd_alter_table ‘PhotoObj ADD t INT’, 1000 sd_create_index ‘run_index ON Photoobj (run)’ sd_drop_index ‘PhotoObj.run_index’ Scalable Table Removal sd_drop_table ‘PhotoObj’ IASTED-DBA 2006, Innsbruck, Feb 15th, 2006

SD-SQL Server Application Interface Images Secondary Image Creation sd_create_image ‘Ceria’, ‘PhotoObj’ sd_create_image ‘Ceria2’, ‘PhotoObj’ Secondary Image Removal sd_drop_image 'PhotoObj’ IASTED-DBA 2006, Innsbruck, Feb 15th, 2006

SD-SQL Server Application Interface Scalable Queries USE Skyserver /* SQL Server command */ Scalable Update Queries sd_insert ‘INTO PhotoObj SELECT * FROM Ceria5.Skyserver-S.PhotoObj’ Scalable Search Queries sd_select ‘* FROM PhotoObj’ sd_select ‘TOP 5000 * INTO PhotoObj1 FROM PhotoObj’, 500 IASTED-DBA 2006, Innsbruck, Feb 15th, 2006 IASTED - DBA 2006, Innsbruck, February 14-16, 2006

SD-SQL Application Interface IASTED-DBA 2006, Innsbruck, Feb 15th, 2006

SD-SQL Application Interface Command Processing SD-SQL Server uses distributed stored procedures with dynamic SQL… SD-SQL Server processes every command as a distributed transaction at Repeatable Read isolation level See details in the papers “SD-SQL Server: a Scalable Distributed Database System”, CERIA Research Report 2005-12-13, December 2005  “Overview of Scalable Distributed Database System SD-SQL Server”, Intl. Workshop on Distributed Data and Structures, WDAS 2006, Santa Clara, CA, Carleton Scientific. IASTED-DBA 2006, Innsbruck, Feb 15th, 2006

IASTED-DBA 2006, Innsbruck, Feb 15th, 2006 Conclusion Scalable distributed databases with scalable tables are now a reality with SD-SQL Server No more manual repartitioning Unlike in any other DBS we know about See the “Related Work” in the paper The performance analysis proves Efficiency of our design Immediate utility of SD-SQL Server IASTED-DBA 2006, Innsbruck, Feb 15th, 2006

For more details: http://ceria.dauphine.fr Thank you.