Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 1\46 pages Soror SAHRI Design and Implementation of a Scalable Distributed Database System: SD-SQL Server Thesis Presentation – CERIA Laboratory

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 2\46 pages 1. Introduction 2. State of The Art 3. SD-SQL Server Architecture 7. Conclusion & Future Work 5. Implementation of SD-SQL Server 4. SD-SQL Server Application Interface PLANPLAN 6. Performance Measurements

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 3\46 pages Facts Objectve Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Facts  Most of DBSs have distributed/parallel versions  SQL Server, Oracle, DB2…  DBSs do not provide dynamically scalable tables  All require manual repartitioning when tables scale-up

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 4\46 pages Facts Objective  Scalable Distributed Partitioning of Relational Tables Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Scalable Distributed Database System SD-SQL Server

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 5\46 pages 3. SD-SQL Server Architecture 5. Implementation of SD-SQL Server 4. SD-SQL Server Application Interface 1. Introduction 2. State of the Art PLANPLAN 7. Conclusion & Future Work 6. Performance Measurements

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 6\46 pages Parallel DBMSs SDDSs Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work  Oracle 10g [LB05] [LB05] K, Loney & B, Bryla. Oracle Database 10g, DBA Handbook

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 7\46 pages Parallel DBMS SDDSs  SDDSs provide many scalable distributed partitioning schemes  LH*, RP*, k-RP*, LH*RS…  These schemes can serve as the basis for an SD-DBS architecture Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work  An SDDS is a new class of data structures  Specific for multicomputers, P2P, Grids…  Why ?

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 8\46 pages 3. SD-SQL Server Architecture 5. Implementation of SD-SQL Server 4. SD-SQL Server Application Interface 1. Introduction 2. State of the Art PLANPLAN 7. Conclusion & Future Work 6. Performance Measurements

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 9\46 pages Overview Nodes, NDBs & SDBs Scalable Tables Images SD-SQL Server?  SD-SQL Server is a Scalable Distributed Database System (SD-DBS)  SD-SQL Server uses the reference architecture  Proposed by Pr. Litwin, Pr. Schwartz & Pr. Risch  2nd Intl. Workshop on Cooperative Internet Computing, 2002  SD-SQL Server is based on the RP* SDDS principles  SD-SQL Server runs on Microsoft SQL Server 2000 Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 10\46 pages Gross Architecture Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Overview Nodes, NDBs & SDBs Scalable Tables Images Linked SQL Servers N1 N2NiNi+1 SSPC User/Application sd_alter_table User/Application sd_insert Split [Litwin & Sahri. WDAS 2004] NDBs SD-SQL server SD-SQL client SD-SQL Server Managers SD-SQL peer The SD-SQL Server originality ?  The automatic extension of the scalable tables  of their NDBs  of their SD-SQL Server nodes

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 11\46 pages The Nodes, NDBs & SDBs Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Overview Nodes, NDBs & SDBs Scalable Tables Images DB1 NDB Node1Node2Node3Nodei DB1 NDB …… DB1 NDB MDB DB1 SDB DB2 SDB DB2 NDB

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 12\46 pages Scalable Tables  A scalable (distributed) table is a collection of segments  Segments are SQL tables  A scalable table has, initially, only one primary segment  At some server or peer NDB  All the segments of a scalable table have the same scheme Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Overview Nodes, NDBs & SDBs Scalable Tables Images

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 13\46 pages Scalable Tables: Meta-data  Each scalable table has meta-data:  The segment capacity  The actual partitioning of the scalable table  The check constraint of each segment Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Overview Nodes, NDBs & SDBs Scalable Tables Images A check constraint defines the Min and Max for each segment  These meta-data are stored in the meta-tables  excluding the check constraints

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 14\46 pages ……. DB1 SDB Scalable Tables: Meta-data N1.DB1N2.DB1N3.DB1 T Scalable Table Ni.DB Size N1.DB1 Primary Ni.DB1 Nodes N1.DB1 N2.DB1 N3.DB1 RP Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Overview Nodes, NDBs & SDBs Scalable Tables Images Meta-Tables

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 15\46 pages Scalable Tables: Splitting Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Overview Nodes, NDBs & SDBs Scalable Tables Images  The number of segments in a scalable table is variable  A segment that overflows splits  A split occurs when an insert overflows the segment capacity  Every split produces one or more new segments for a scalable table

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 16\46 pages Scalable Tables: Splitting Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Overview Nodes, NDBs & SDBs Scalable Tables Images S b+1 S S1 p b+1-p p=INT(b/2) C( S)=   { c: c < h = c (b+1-p)} C( S 1 )={c: c > = c (b+1-p)} Check Constraint? b SELECT TOP P * INTO Ni.Si FROM S ORDER BY C ASC SELECT TOP P * WITH TIES INTO Ni.S1 FROM S ORDER BY C ASC

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 17\46 pages DB1 SDB T Scalable Table sd_insert N1N2N4N3 NDB DB1 sd_insert NDB DB1 Ni sd_create_node sd_insert N3 NDB DB1 sd_create_node_database NDB DB1 ……. sd_create_node_database Scalable Tables: Splitting Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Overview Nodes, NDBs & SDBs Scalable Tables Images DB1 SDB

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 18\46 pages Images Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Overview Nodes, NDBs & SDBs Scalable Tables Images  An Image hides the scalable table partitioning  An image is an SQL Server distributed updateable partitioned view of the table  An SQL Server Union-all view with check constraints  An image resides on client or peer NDBs

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 19\46 pages Image Types  Primary image  Resides at the creation node  Has the name of the scalable table  Secondary images  Reside at other client or peer NDBs of the SDB  Have a specific name, other than that of the table  To avoid name conflict Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Overview Nodes, NDBs & SDBs Scalable Tables Images

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 20\46 pages Image Adjustment Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Overview Nodes, NDBs & SDBs Scalable Tables Images  An image presents the actual partitioning of its scalable table  Defines the partitioning as known to the client  It do not address any new segments resulted from a split  Are dynamically adjustable by the client  When a query to the image comes in  Image checking  Image adjustment if necessary

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 21\46 pages Image Adjustment Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Overview Nodes, NDBs & SDBs Scalable Tables Images  Get the number of segments presented in the image, N1  Get the number of segments of the scalable table, N2  Compare N1 and N2:  If N1<N2 then Image Adjustment  Alter the partitioned view definition

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 22\46 pages Images: Example N1.DB1 N2.DB1 N3.DB1 T Scalable Table Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Overview Nodes, NDBs & SDBs Scalable Tables Images CREATE VIEW T AS SELECT * FROM N2.DB1.SD._N1_TCREATE VIEW T AS SELECT * FROM N2.DB1.SD._N1_T UNION ALL SELECT * FROM N3.DB1.SD._N1_T UNION ALL SELECT * FROM N4.DB1.SD._N1_T Primary Image DB1 SDB N4.DB1 T Image

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 23\46 pages 3. SD-SQL Server Architecture 5. Implementation of SD-SQL Server 4. SD-SQL Server Application Interface 1. Introduction 2. State of the Art PLANPLAN 7. Conclusion & Future Work 6. Performance Measurements

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 24\46 pages Principles  The application interface manipulates scalable tables through SD-SQL Server commands  The SD-SQL Server commands start with ‘sd_’ to distinguish from SQL Server commands for static tables Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Principles Nodes Management SDBs & NDBs Management Scalable Tables & Images Management Scalable Queries Management INSERT sd_insert CREATE TABLE sd_create_table [Litwin, Schwartz & Sahri. IASTED-DBA 2006]

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 25\46 pages Nodes Management  Node Creation  sd_create_node ‘Dell1’ /* Server by default */  sd_create_node ‘Ceria’, ‘client’  Node Alteration  sd_alter_node ‘Ceria’, ‘ADD server’ /* Becomes peer*/  Node Removal  sd_drop_node ‘Ceria’ Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Principles Nodes Management SDBs & NDBs Management Scalable Tables & Images Management Scalable Queries Management

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 26\46 pages SDB & NDB Management  SDB Creation  sd_create_scalable_database ‘SkyServer’, ‘Dell1’, ‘Server’, 2 /* Creates the primary SkyServer NDB as well at Dell1*/  SDB Alteration  sd_create_node_database ‘SkyServer’, ‘Ceria’, ‘Client’  SDB Removal  sd_drop_scalable_database ‘SkyServer’ Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Principles Nodes Management SDBs & NDBs Management Scalable Tables & Images Management Scalable Queries Management

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 27\46 pages Scalable Tables Management  Scalable Table Creation  sd_create_table ‘PhotoObj (Objid BIGINT PRIMARY KEY..)’,  Scalable Table Alteration  sd_alter_table ‘PhotoObj ADD t INT’, 1000  sd_create_index ‘run_index ON Photoobj (run)’  sd_drop_index ‘PhotoObj.run_index’  Scalable Table Removal  sd_drop_table ‘PhotoObj’ Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Principles Nodes Management SDBs & NDBs Management Scalable Tables & Images Management Scalable Queries Management

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 28\46 pages Images Management  Secondary Image Creation  sd_create_image ‘Ceria’, ‘PhotoObj’  Secondary Image Removal  sd_drop_image 'PhotoObj’ Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Principles Nodes Management SDBs & NDBs Management Scalable Tables & Images Management Scalable Queries Management

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 29\46 pages Scalable Queries Management USE SkyServer /* SQL Server command */  Scalable Update Queries  sd_insert ‘INTO PhotoObj SELECT * FROM Ceria.Skyserver-S.dbo.PhotoObj’  Scalable Search Queries  sd_select ‘* FROM PhotoObj’  sd_select ‘TOP 5000 * INTO PhotoObj1 FROM PhotoObj’, 500 Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Principles Nodes Management SDBs & NDBs Management Scalable Tables & Images Management Scalable Queries Management

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 30\46 pages 3. SD-SQL Server Architecture 5. Implementation of SD-SQL Server 4. SD-SQL Server Application Interface 1. Introduction 2. State of the Art PLANPLAN 7. Conclusion & Future Work 6. Performance Measurements

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 31\46 pages Image Binding Internal Processing Concurrency Experiments Command Processing  Let Q a scalable query using the PhotoObj image:  sd_select ‘COUNT (*) FROM PhotoObj’ Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Find Images in Q PhotoObj Image Adjustment Execution of Q [Litwin, Schwartz & Sahri. WDAS 2006]

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 32\46 pages Concurrency Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Internal Processing Concurrency Experiments  SD-SQL Server processes every command as SQL distributed transaction at Repeatable Read isolation level  Much less blocking than at Serializable Level  SD-SQL Server performs the split asynchronously with the insert that triggered it  It launches the actual splitting as an asynchronous job called splitter

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 33\46 pages Concurrency Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Internal Processing Concurrency Experiments  Splits use exclusive locks on segments and on tuples in RP meta-table.  Shared locks on other meta-tables: Primary, NDB meta-tables  Scalable queries use basically shared locks on meta-tables and any other table involved

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 34\46 pages Concurrency: example Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Internal Processing Concurrency Experiments Splitter sd_alter_table Dell1.SkyServer Dell3 Dell2 Dell1 RP PhotoObj Exclusive Lock Waiting Exclusive Lock Shared Lock Exclusive Lock X X

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 35\46 pages 3. SD-SQL Server Architecture 5. Implementation of SD-SQL Server 4. SD-SQL Server Application Interface 1. Introduction 2. State of the Art PLANPLAN 7. Conclusion & Future Work 6. Performance Measurements

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 36\46 pages Experimental Environment Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Internal Processing Concurrency Experiments  6 Machines Pentium IV 1.7 GHz  RAM: 780 Mb & 1 Gb  Operating System: Windows 2K Server  Ethernet Network: max bandwidth of 1 Gb/s  Use of SQL Analyzer for editing queries  Use of SQL Profiler to take measurements

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 37\46 pages  We use SkyServer database as benchmark  Provided and installed at Ceria by Dr. Gray  SkyServer brings the entire database of the Sloan Digital Sky Survey, SDSS  We use of the PhotoObj table as an example scalable table  In our experiments, PhotoObj has 158,426 tuples (about 260 MB)  Originally, it has 14 M tuples The SkyServer Benchmark Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Principles Nodes Management SDBs & NDBs Management Scalable Tables & Images Management Scalable Queries Management

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 38\46 pages Splitting Measurements Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Internal Processing Concurrency Experiments [Litwin, Sahri & Schwartz. WDAS 2004] Splitting of PhotoObj scalable table into 2, 3, 4 and 5 segments according to different capacities

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 39\46 pages Image Adjustment Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Internal Processing Concurrency Experiments (Q1): sd_select ‘TOP 10 objid FROM PhotoObj WHERE objid not in (SELECT objid FROM PhotoObj WHERE objid Query (Q1) execution time

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 40\46 pages Comparison between SD-SQL Server and SQL Server (Q2): sd_select ‘COUNT (*) FROM PhotoObj’ Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work Internal Processing Concurrency Experiments Execution time of (Q2) on SQL Server and SD-SQL Server [Litwin, Sahri & Schwartz. BNCOD 2006]

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 41\46 pages 3. SD-SQL Server Architecture 7. Conclusion & Future Work 5. Implementation of SD-SQL Server 4. SD-SQL Server Application Interface 1. Introduction 2. State of the Art PLANPLAN 6. Performance Measurements

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 42\46 pages Conclusion Future Work Conclusion Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work  Scalable distributed databases with scalable tables are now a reality with SD-SQL Server  No more manual repartitioning  Unlike in any other DBS we know about  The performance analysis proves  Efficiency of our design  Immediate utility of SD-SQL Server

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 43\46 pages Conclusion Future Work  More performance measurements  With the SDSS queries  With the SkyServer benchmark of 80 Gb size  Error processing  Management of fault tolerance  Use of the high availability methods Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 44\46 pages Conclusion Future Work  Application on other DBMSs  Oracle, DB2, etc.  Use of the SD-SQL Server principles on P2P systems or Grid Computing  Use of SD-SQL Server as core component of a virtual repository of eGov documents Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 45\46 pages Thanking  Work partly supported by  CEE Project eGov  MS Research  CEE Project ICONS Conclusion Future Work Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work

Soror SAHRI – June 13 th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server 46\46 pages Soror SAHRI Conclusion Future Work Introduction State of the Art SD-SQL Server Architecture SD-SQL Server Application Interface Implementation of SD-SQL Server Conclusion & Future Work