1. 701 - Building with the Data Tools project John Graham Der Ping Chou

2. Data Tools Platform DTP New top-level project at Eclipse Managed by Sybase, IBM, and Actuate Additional contributions: –Versant –ObjectNation GmbH –Pervasive Software Broad community support

3. Who We Are John Graham DTP PMC Sybase Der Ping Chou WTP Committer DTP IBM

4. Outline Overview of DTP Model Base project detail Case study: Demonstration of extending the base models Questions

5. Purpose of DTP Provide a comprehensive set of extensible data frameworks exemplary tools for data-centric applications

6. Why DTP? Data is fundamental to software applications Known set of requirements based on years of experience Unify the approach for Eclipse Enable higher-level data applications

7. Scope of DTP PMC worked very hard on this section The Scope section was very difficult: –Data is used everywhere –How to draw the line between what naturally belongs in DTP and what does not? Settled on a test based on problem domain

8. Scope Test Data-centric applications –Associated with a data source –Having a problem domain no more specific than just “data” within the Eclipse ecosystem of projects Data-dependent applications –Built on data-centric frameworks and tools –Have a more specific problem domain in the Eclipse ecosystem of projects

9. Example: In Scope SQL Editor Associated with a data source –SQL statement itself –Database system using the statement Problem domain –Generic (relational) data operations –Not naturally in scope for other, existing Eclipse projects

10. Example: Out of Scope Java source code editor: Uses data (source code) Problem domain is Java tooling –Eclipse has the Java Development Tools (JDT) project –So, not in scope for DTP, because the more specific JDT problem domain includes it

11. DTP Structure As a top-level project at Eclipse: DTP project –(Sub)project Component DTP is managed by the PMC A project is managed by a project lead A component has a technical lead All volunteers, all open source

12. Current Data Tooling Web Tools Platform (WTP) project has an rdb component Business Intelligence and Reporting Tools Project (BIRT) Eclipse plug-ins, both commercial and open source

13. Consolidation in DTP rdb moving components to DTP BIRT providing Open Data Access (ODA) to DTP Sybase open sourcing connectivity and SQL tooling for DTP Donations from ObjectNation GmbH Other areas under discussion

14. Possible Future Components In support of Object/Relational Mapping (ORM): –Versant, Oracle have projects at Eclipse in this area –Working with them, and others, to insure that DTP meets the community’s data requirements XML/Relational Mapping Administration Extract-Transform-Load (ETL)

15. First DTP Release Concentrating on relational structures Milestones planned for –2 in 5Q4 –2 in 6Q1 Release in late 6Q1 Project plan draft posted on DTP site

16. Milestone Components

17. DTP Architecture

18. DTP Projects Model Base Connectivity SQL Development Tools

19. Model Base Project Provides foundational domain models for DTP and DTP consumers, based on Eclipse Modeling Framework (EMF) and using a model-driven development with UML

20. Model Base Benefits Support for change management Broad editing support: visual and otherwise Extensible, database agnostic models Compliance with latest standards, such as SQL Supporting JDBC and other connection standards

21. Model Base Components Driver Definition model SQL model (99/03) Database Definition model SQL Query model

22. Connectivity Project Provides foundational driver and connection support for data-driven application build in Eclipse.

23. Connectivity Benefits One place to define drivers One place to define (configure) connections Unified view of data sources Unified view of data

24. Connectivity Components Driver Management Connection Profiles Data Source Explorer JDBC Connection Support Open Data Access

25. Driver Management Allows for driver management in Eclipse Template-based Generic Extensible

26. Connection Profiles Abstraction of “connection” Configuration details Generic, not just for databases Extensible for –Types –Content

27. Data Source Explorer Exemplary code to demonstrating how to Populate the view from the SQL Model. Use the Navigator View framework to display database in Database Explorer (WTP)

28. Data Source Explorer cont.. Use the DDL Generator to generate DDL Extract from and load data into a relational table

29. Data Source Explorer cont.. Filter support on database elements – schema, table, view…

30. Data Source Explorer cont.. Connect, disconnect, execute/launch SQL Scrapbook and work offline

31. Data Source Explorer cont.. Sample, open/update, extract from and load data into a relational table

32. JDBC Connection Support Exemplary code that demonstrates Supporting a generic JDBC catalog loader Extending the wizard framework to support multiple drivers for a single db vendor Specialization of the catalog loading mechanism

33. Open Data Access (ODA) Previously in BIRT project Provides abstraction over data, like Connection Profile does for connections Define “data drivers” for data types Meta-data access support

34. SQL Dev Tools Provides a set of frameworks and exemplary tools for using SQL.

35. SQL Dev Tools Benefits How many text editors do you have to write? –How many SQL editors does Eclipse need? –How about visualization of SQL? Provide a common SQL Query Model framework –Extension of SQL Model –Detailed EMF model representation of SQL query Covers DML: Select, Insert, Update, Delete –Based on SQL- 2003 with vendor-specific extensions –UML Model -> EMF Utilities -> Generated Code –Extensible SQL source generation Support a SQL DML parser –Based on LPG parser generator –LPG is also used to create the Eclipse Java parser –Targets SQL Query Model directly as parse result –Uses Eclipse extension-point mechanism to support multiple SQL dialects

36. SQL Query Model/Parser Architecture Client SQL Parser SQL Query Model

37. SQL Dev Tools Components Frameworks for Routines SQL Query parser SQL Execution Plan

38. SQL Dev Tools Components Exemplary tools SQL Editor Visual SQL Editor Results View Script History

39. Routines A “routine” is –Stored procedure –Function Generic, vendor-neutral –Creation, editing, saving –Execution –Debugging

40. SQL Query Parser Complies with SQL 03 Based on LPG parser generator –LPG is also used to create the Eclipse Java parser Targets SQL Query Model directly as parse result Uses Eclipse extension-point mechanism to support multiple SQL dialects

41. SQL Execution Plan Execution Plans important for –Understanding database performance –Tuning queries Framework enables –Generic access to execution plans –Ability to visualize in text and (later, perhaps) graphical means

42. SQL Editor Eclipse text editor for SQL –Syntax colorization –Code assist –Templates Syntax checks and feedback Dialect specialization through underlying SQL Query model

43. Visual SQL Editor Graphical representation of SQL statements Ability to construct statements without worrying (as much) about SQL syntax As with SQL Editor, dialect specialization supported Smaller brother of “Visual Query Builder” notion floated earlier in DTP

44. Results View Display results of SQL execution –Uses a Connection Profile instance –Uses a SQL statement Sorting, filtering of view Export of results

45. Script History Stores queries executed during an Eclipse session Allows for running queries multiple times Thinking about filters based on Connection Profile instances Thinking about persistence across platform instances

46. In Depth: The Model Base Project

47. Model Base Themes Using UML to model SQL Model and Database Definition Model, SQL Query Model to support EMF for DTP Published Model: http://www.eclipse.org/webtools/wst/components/rdb/WebPublishedSQLModel/SQLMo del.htm http://www.eclipse.org/webtools/wst/components/rdb/WebPublishedDBDefinitionModel/ DBDefinition.htm

48. Model Base Themes - 1 Tightly integrated and loosely coupled Support the latest SQL Standard

49. Model Base Themes - 2 Exemplary application built on top of the model to exhibit generic implementation vs. Vendor specific extension

50. Model Base Themes - 3 Minimum support vs. rich database element support Future model base upgrade

51. Case Study Supporting richer database features in DTP model framework

52. Case Study Adding new database element support to SQL Model –Change the SQLModel (Rose model) file to add your database element –Use the Eclipse framework to generate a new EMF model –Generate code to create a new Database Definition Model Generate db vendor document using the provided utility Provide databaseDefinition and uri_mapping extension points a database definition plugin Extend database catalog loader – –Adding class file to handle newly added database elements Extend database DDL Generator – Adding database UI plug-in to support Vendor specific JDBC connection –

53. Contacting DTP Web site: www.eclipse.org/datatools Newsgroup: news://news.eclipse.org/eclipse.dtp Mailing lists: www.eclipse.org/mail/main.html#datatools Wiki : http://eclipsewiki.editme.com/DataTools Conferences: –EclipseWorld –EclipseCon –…

54. Questions?