A JDBC driver supporting Data Source Integration Jian Jia

Motivation Manually integrate information sources is painful, because of – Heterogeneous data source – inconsistent / incomplete data structure information – Platform dependency

A Solution Unity – Automate integration process – Uses ODBC to access multiple data sources – X-Spec to capture semantic meaning of the data --- Standard dictionary – In C++ – On Windows platform

Goal of this Project Unity JDBC Driver – Embed integration function of Unity into a JDBC driver – X-Spec as the dictionary – in Java – platform independent – Access multiple data sources through JDBC

Migrating Unity from ODBC to JDBC Integration Module JDBC … DB1DB2DBn … ResultsSemantic Queries User Queries SQL Unity JDBC Driver User Queries Integration Module ODBC … DB1DB2DBn … ResultsSemantic Queries SQL Unity

Basic classes of JDBC API DriverManager Driver Connection StatementResultSet ResultSetMetaData registers provides createsretrieves provides

JDBC Driver Types JDBC-ODBC Bridges plus ODBC drivers Native-API partly-Java drivers JDBC-Net pure Java drivers Native-protocol pure Java drivers

Semantic Query An example SELECT [Employee].id, [Department;Employee].name WHERE [Employee].age > 30 –All Fields/Tables that have the same semantic meaning should have same semantic name. –Semantic query refers a field by its semantic name. –There is no explicit relation specifications(from table, join, union) in the query. –X-Spec Document stores information about all semantic names and corresponding system names for every field/table. –No nested query. –Semantic Query should be parsed to create sub-query in standard form(SQL’92) for each data source.

Semantic Query Grammar SELECT ALL DISTINCT [Column], FROM Tables WHERESearch Condition GROUP BY Columns ORDER BY [Column], DESC /ASC

Semantic Query Grammar Search Condition Expression[NOT] LIKE“[%] String” Column IS [NOT] NULL (Expression) Expression = <   > <> Expression ORAND NOT

Parsing X-Spec X-Spec Table 1 X-Spec Table 2 X-Spec Table 3 X-Spec Table n X-Spec Field 1X-Spec Field 2X-Spec Field k ……. X-spec Key 1X-spec Key 2X-spec Key j … X-spec Joins … … … …

Parsing Semantic Query Query Translator Semantic query S_listC_listF_listGroupBy_listOrderBy_list PASS ONEPASS TWO Selected Fields (Sys_Name) Used for integration Selected X-Spec Fields Mapping semantic Name to System Name; Build sub-query Sub Query 1Sub Query 2Sub Query n ……..

Sub-Query Generation S-List(Selection-List) Only those semantic fields that are in the data source can be substituted by their system names, and added to corresponding sub- query selection list C-List(Condition-List) An expression can only be added to sub-query condition list only when all semantic arguments are in the data source.

Sub-Query Generation S-List C-List OrderBy-List Sub-S-Clause Sub-From- Clause Sub-Where- Clause Sub-Order By- Clause

Inside JDBC Driver Semantic query Selected Fields (Sys_Name) Sub Query 1 Sub Query 2 Sub Query n … Query Translator DB1DB2DBn … JDBC 1JDBC 2JDBC n ResultSet 1 ResultSet 2 ResultSet n ResultSet- MetaData ResultSet- MetaData ResultSet- MetaData Join Union ResultSet MetaData Integration

Integration Method JOIN Merge Join by Global Keys MultiValue Field – Data inconsistent UNION Simply append one ResultSet to another Do not need match keys

A Simple Example Semantic Query (for two data sources) SELECT F1, F2, F3, F4, F5 WHERE C1 AND C2 SELECT f1a, f3a, f5a FROM tableA WHERE C1 SELECT f1b,f2b, f4b FROM tableB WHERE C2 Sub-query for DB1 Sub-query for DB2

Example Result Join Result Assume Key_A and Key_B are two system names of the global KEY in two data sources DB 1DB 2

Example Result Union DB 1DB 2

Future Work Operations cross data sources Complete Algorithms for Result Integration Automated Updates on heterogeneous data Sources Implement Group By, From in semantic query

??????