TopLink: Now and in the future, O-R Mapping Product of Choice Doug Clarke Principal Product Manager, Oracle9iAS TopLink Oracle Corporation

Speaker’s Qualifications  Doug has a decade of experience in OO Persistence – Development, Consulting & Training  Product Manager for TopLink product

Underestimation Managing persistence related issues is the most underestimated challenge in enterprise Java today – in terms of complexity, effort and maintenance

Agenda  Ways to access RDB from J2EE  Impedance Mismatch  TopLink Overview – TopLink in Development – Runtime Architecture – O-R Mapping Flexibility – Queries – Transactions – Caching – Locking – Performance Features  TopLink Roadmap

J2EE Access of Relational Data  Direct JDBC – Direct SQL calls – Use rows and result sets directly  Persistence Layer – Accessed as objects or components – Transparent that the data is stored in RDB – Persistence layer in middle tier handles object- relational mapping and infrastructure – Required if doing business logic in the middle tier! – Focus of this Session

Direct JDBC  JDBC is NOT a persistence framework – JDBC is a database connection utility – Ok for “Window on data” style application – Ok when business logic is entrenched on database – J2EE becomes a presentation layer… SQLrows JDBC

Object Persistence Layer  Abstracts persistence details from the application layer SQLrows Objects Persistence Layer Objects object-level querying and creation results are objects results are returned as raw data API uses SQL or database specific calls object creation and updates through object-level API J2EE & Web Services JDBC

Problem  Building Java applications with relational databases is a VERY challenging and labor intensive problem to solve – Fundamentally different technology – Different skill sets – Different staff/ownership – Different modeling and design principles Differences must be resolved to fulfill business requirements ?

Impedance Mismatch FactorJ2EERelational Databases Logical View Of Data Objects, methods, inheritance Tables, SQL, stored procedures ScaleHundreds of megabytesGigabytes, terabytes RelationshipsMemory referencesForeign keys UniquenessInternal object idPrimary keys Key SkillsJava development, object modeling SQL, Stored Procedures, data management ToolsIDE, Source code management, Object Modeler, etc Schema designer, query manager, database configuration, etc

Object Level Options  Depends on what component architecture is used: – Entity Beans BMP – Bean Managed Persistence – Entity Beans CMP – Container Managed Persistence – POJO – “Plain Ol’ Java Objects” via Persistence Layer  May be “home-grown” following “DAO” Data Access Object patterns  May use JDO – “Java Data Objects”  May use off the shelf products

J2EE Architectures  J2EE Architecture Options – Servlets – JSP – Session Beans – Message Driven Beans – Web Services  Bottom Line – Java application needs to access relational data somehow…

Oracle9iAS TopLink Solution Oracle9iAS TopLink provides a proven, powerful “out of the box” solution to address the diversity between Java and Databases Oracle9iAS TopLink

TopLink in Development IMPORT GENERATE IMPORT (.class) GENERATE (.java) Java IDE TopLink Meta-Data (XML) Java classes Relational database (schema) TopLink Mapping Workbench

TopLink Runtime Architecture Data Source TopLink Application Application Server Session Data Access Mappings JDBC J2EE Container JTA CMP/ BMP Cache Query TX Entities Java Objects EJB Entity Beans JSP, Servlet, Struts, etc. App Logic

Basic J2EE Persistence Checklist  O-R Mappings  Runtime Components – O-R Mapping Flexibility – Queries – Transactions – Caching – Locking – Performance Features

Data and Object Models  Rich, flexible mapping capabilities provide data and object models a degree of independence  Otherwise, business object model will force changes to the data schema or vice-versa  Often, J2EE component models are nothing more than mirror images of data model or vice versa – NOT desirable

Mapping Tools  The underlying mapping flexibility is very important

Simple Object Model Customer id: int name: String creditRating: int Address id: int city: String zip: String 1:1 Relationship

Typical 1-1 Relationship Schema CUST IDNAME A_ID C_RATING ADDR IDCITYZIP

Other possible Schemas… CUST IDNAMEC_RATE C_ID ADDR IDCITYZIP CUST IDNAMEC_RATING C_ID ADDR IDCITYZIP A_ID CUST_ADDR C_ID CUST IDNAMECITY ZIPC_RATING

Even More Schemas… CUST IDNAME A_ID ADDR IDCITYZIP CUST_CREDIT ID C_RATING CUST IDNAME ADDR IDCITYZIP CUST_CREDIT ID C_RATING A_ID CUST IDNAME ADDR IDCITYZIP CUST_CREDIT ID C_RATING A_ID CC_ID CUST IDNAME CUST_CREDIT IDC_RATING ADDR IDCITYZIPC_ID CUST IDNAME CUST_CREDIT ID C_RATING ADDR IDCITYZIPC_ID

Benefits of Good O-R Mapping  Flexible - Just showed nine valid ways a 1-1 relationship could be represented in a database (Most persistence solutions will only support one)  Without good support, designs will be forced  Comprehensive support for custom types, inheritance,  Simple mapping for identity management  Custom type support  Non-intrusive/transparent to the business model developer

MORE than just Mapping  Querying  Transactions  Caching  Concurrency  Performance

Queries  Java developers are not usually SQL experts – Maintenance and portability become a concern when schema details hard-coded in application  Allow Java based queries that are translated to SQL and leverage database options employee.get(“manager”).get(“address”) = someAddress SELECT * FROM EMP t1, EMP t2, ADDR t3 WHERE t1.MGR_ID = t2.EMP_ID AND t2.ADDR_ID = t3.ADDR_ID AND t3.ADDR_ID = Generates …

Query Features  Ability to trace, profile, and tune SQL  Ad hoc SQL where necessary  Must be able to leverage database abilities – Outer joins – Nested queries – Stored Procedures – Database Specific Optimizations (Oracle Hints)  Choice of query language: SQL, Stored procedures, EJBQL, or TopLink’s Object expressions  In-Memory querying and transaction conformity

Transaction Management  J2EE apps typically support many clients sharing small number of db connections  Ideally would like to minimize length of transaction on database Time Begin Txn Commit Txn Begin Txn Commit Txn

Support for Referential Integrity  Java developers manipulate object model in a manner logical to the business domain  May result in ordering of INSERT, UPDATE and DELETE statements that violate database constraints  Persistence layer should automatically manage this and allow options for Java developer to influence order of statements

Object Cache  Cache your business objects  Improved performance by reducing reads  Configurable  Support for clustered caches  Concurrency support for multi-client reading and writing

Caching in the real world  Any application that caches data, now has to deal with stale data  When and how to refresh?  Will constant refreshing reduce performance?  Problem is compounded in a clustered environment  App server may want be notified of database changes TopLink supports all of these cases

Caching OO Query SQL Query (if desired) Results(s) Does PK for row exist in cache? YES – Get from Cache NO – Build bean/object from results Return object results

Concurrency: Locking  J2EE Developers want to think of locking at the object level  Databases may need to manage locking across many applications  Persistence layer or application server must be able to respect and participate in locks at database level

Optimistic Locking  DBA may wish to use version, timestamp and/or last update field to represent optimistic lock – Java developer may not want this in their business model – Persistence layer must be able to abstract this  Must be able to support using any fields including business domain

Pessimistic Locking  Requires careful attention as a JDBC connection is required for duration of pessimistic lock  Should support SELECT FOR UPDATE [NOWAIT] semantics  Consider Application level pessimistic locking Time Begin Txn Commit Txn Begin Txn Commit Txn Pess Lock

Performance: Lazy Reads  J2EE Applications work on the scale of a few hundreds of megabytes  Relational databases routinely manage gigabytes and terabytes of data  Persistence layer must be able to transparently fetch data “just in time”

Just in Time Reading – Faulting Process Customer Orders Proxy Accessing relationship for first time Get related object based on FK SQL if not cached Check Cache Plug result into Proxy Orders

Object Traversals  Even with lazy reads, object traversal is not always ideal – To find a phone number for the manufacturer of a product that a particular customer bought, may do several queries:  Get customer in question  Get orders for customer  Get parts for order  Get manufacturer for part  Get address for manufacturer – Very natural object traversal results in 5 queries to get data that can be done in 1

N+1 Reads Problem  Many persistence layers and application servers have an N+1 reads problem  Causes N subsequent queries to fetch related data when a collection is queried for  A side effect of the impedance mismatch and poor mapping and querying support in persistence layers

N+1 Reads Problem Pool of Created Objects or Beans Persistence Layer or EJB Container findByCity() 1 2 For each Customer Fetch their Address Address n Returns collection n If Address had related objects, they too may be fetched 2n+1 Reads! Container returns results C C CC

N+1 Reads  Must have solution to minimize queries  Need flexibility to reduce to 1 query, 1+1 query or N+1 query where appropriate – 1 Query when displaying list of customers and addresses – known as a “Join Read” – 1+1 Query when displaying list of customers and user may click button to see addresses – known as a “Batch Read” – N+1 Query when displaying list of customers but only want to see address for selected customer

Performance and Tuning Options  Minimal Writes, Updates  Batch Reading, Writing  SQL ordering  Transformation support  Existence checks  Stored procedures  Read Lock Support  Statement Caching  Scrolling cursors  Partial Attribute Queries  “Just in Time” reading AND MUCH MORE!  Automatic change detection  Caching policies and sizes  Parameterized SQL  Pre-allocation of sequence numbers  Cache synchronization support  Optimistic, Pessimistic locking  Joining object retrieval optimization  In memory querying  Dynamic queries  Query optimization

Why is TopLink the O-R leader?  Productive development tools  Extremely flexible O-R mappings  Choice and performance in querying  Scalable cache and transaction framework  Powerful performance tuning options  Strong J2EE support and integrations  Complete architecture and database flexibility Award winning product based on years of real world application development and backed by world’s leading enterprise software company.

What’s Next: TopLink Roadmap  Maintain leadership in O-R persistence  Expand data source support: – EIS through J2EE Connector Architecture – Object-to-XML  Maintain current with standards and provide functionality demanded by our customers

TopLink & XML  XML documents can be represented at different levels of abstractions in J2EE applications: – Parsed document (DOM, SAX …) - parser – Unmanaged Java objects from non-transactional data source – data converter – Managed Java objects from a transactional data source – persistence manager

3 Levels of XML Representation XML Parser O-X Data Converter XML Document Unmanaged Object Persistence Manager XDBEIS JDBCJ2C DOM Managed Object/EJB File Web Service BPMJMS

Impact on J2EE Integration Solutions  This technology will be critical for many J2EE applications accessing EIS systems via JCA resource adapters – XML based adapters will outgrow proprietary data formats – “80% of the worldwide data resides on mainframes.” Gartner  Without it, developers will have to work with low level XML documents and manually code persistence manager functionality

Benefits of XML/JCA Support  Easier to adopt and integrate web services, disparate data and legacy applications.  Abstracting data details provides flexibility and portability.  Increased developer productivity as they focus on building applications not infrastructure.  Single abstraction layer to access heterogeneous data sources.  Optimized performance and scalability.  Leverage existing architecture and developer skills base

Summary  TopLink is clearly leader in O-R mapping  Many exciting directions for the future

If You Only Remember One Thing… When choosing a persistence layer, Pick one that maximizes your flexibility across all the issues outlined here today!

A Q & Q U E S T I O N S A N S W E R S