1. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 1 - 20/11/2003 C-JDBC: a High Performance Database Clustering Middleware Nicolas Modrzyk Nicolas.Modrzyk@inrialpes.fr

2. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 2 - 20/11/2003 Outline - Motivations  Motivations  Use-Cases  C-JDBC concepts  Performance  Community  Conclusion

3. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 3 - 20/11/2003 Motivations  J2EE performance scalability bounded by database performance  Database tier must be –scalable –fault tolerant (high availability + failover) –without modifying the client application –using open source databases –on commodity hardware

4. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 4 - 20/11/2003 What is

5. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 5 - 20/11/2003 Key ideas  RAIDb controller –gives the view of a single database to the client –balance the load on the database backends  RAIDb levels –RAIDb-0: full partitioning –RAIDb-1: full mirroring –RAIDb-2: partial replication –composition possible

6. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 6 - 20/11/2003 C-JDBC  Middleware implementing RAIDb  Two components –generic JDBC 2.0 driver (C-JDBC driver) –C-JDBC Controller  C-JDBC Controller provides –performance scalability –high availability –failover –caching, logging, monitoring, …  Supports heterogeneous databases

7. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 7 - 20/11/2003 Outline - Use-Cases  Motivations  Use-Cases  C-JDBC concepts  Performance  Community  Conclusion

8. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 8 - 20/11/2003 What C-JDBC offers  Usually, we do this: Database JDBC Driver Application

9. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 9 - 20/11/2003 Pooling Cache Controller Database Virtual Driver What C-JDBC offers  Now we have this: Backend JDBC Driver Application

10. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 10 - 20/11/2003 What C-JDBC offers  And, finally we have all this: Database Virtual Backend Backend recovery Pooling Cache Controller Database Virtual JDBC Driver Load balancing Driver Application

11. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 11 - 20/11/2003 Heterogeneity support  application already written for a specific [commercial] database  user defined rules for on-the-fly query rewriting to execute on heterogeneous backends

12. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 12 - 20/11/2003 Outline - C-JDBC concepts  Motivations  Use-Cases  C-JDBC concepts  Performance  Community  Conclusion

13. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 13 - 20/11/2003 Controller Sockets JMX

14. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 14 - 20/11/2003 Virtual Database  gives the view of a single database  establishes the mapping between the database name used by the application and the backend specific settings  backends can be added and removed dynamically  configured using an XML configuration file

15. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 15 - 20/11/2003 Scheduler  Manages concurrency control  Specific implementations for Single DB, RAIDb 0, 1 and 2  Query-level  Optimistic and pessimistic transaction level –uses the database schema that is automatically fetched from backends

16. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 16 - 20/11/2003 Request cache  caches results from SQL requests  improved SQL statement analysis to limit cache invalidations –table based invalidations –column based invalidations –single-row SELECT optimization  request parsing possible in the C-JDBC driver –offload the controller –parsing caching in the driver

17. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 17 - 20/11/2003 Load balancer  RAIDb-0 –query directed to the backend having the needed tables  RAIDb-1 –read executed by current thread –write executed in parallel by a dedicated thread per backend –result returned if one, majority or all commit –if one node fails but others succeed, failing node is disabled  RAIDb-2 –same as RAIDb-1 except that writes are sent only to nodes owning the written table

18. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 18 - 20/11/2003 Connection Manager  Connection pooling for a backend –Simple : no pooling –RandomWait : blocking pool –FailFast : non-blocking pool –VariablePool : dynamic pool  Connection pools defined on a per login basis –resource management per login –dedicated connections for admin

19. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 19 - 20/11/2003 Recovery Log  Checkpoints are associated with database dumps  Record all updates and transaction markers since a checkpoint  Used to resynchronize a database from a checkpoint  JDBCRecoveryLog –store information in a database –can be re-injected in a C-JDBC cluster for fault tolerance

20. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 20 - 20/11/2003 Octopus integration  Add new backends while being online  Restore dump corresponding to initial checkpoint with Octopus

21. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 21 - 20/11/2003 Outline - Performance  Motivations  Use-Cases  C-JDBC concepts  Performance  Community  Conclusion

22. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 22 - 20/11/2003 TPC-W Performance

23. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 23 - 20/11/2003 RUBiS- Tomcat without C-JDBC caching Tomcat ~50% cpu 1 Database 100% cpu Throughput: 3900 pages/min

24. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 24 - 20/11/2003 RUBiS- Tomcat with C-JDBC caching Tomcat ~55% cpu 1 Database ~20% cpu Throughput: 4200 pages/min C-JDBC <10% cpu

25. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 25 - 20/11/2003 Outline - Community  Motivations  Use-Cases  C-JDBC concepts  Performance  Community  Conclusion

26. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 26 - 20/11/2003 Stats as of November, 2003  Downloads –total : 8500 downloads since may 2003 –last 30 days : > 2800 downloads –> 430.000 hits since first release –2nd most downloaded ObjectWeb project  Mailing lists –c-jdbc@objectweb.org: 101 subscribersc-jdbc@objectweb.org –c-jdbc-commits@objectweb.org: 18 subscribersc-jdbc-commits@objectweb.org  Team –9 committers –1 full-time INRIA engineer

27. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 27 - 20/11/2003 The developer community  Mathieu Peltier (ObjectWeb) –build scripts, automatic installer, JUnit test  Julie Marguerite (ObjectWeb) –JDBCRecoveryLog, automatic schema detection  Christiana Amza (Rice University), Anupam Chanda (Rice University), Sara Bouchenak (EPF Lausanne) –SQL query caching  Guillaume Bort (INRIA Lorraine) –JBoss support  Marek Prochazka (INRIA Rhone-Alpes) –Datasource implementation  Greg Ward (dplanet.ch) –Sybase support, design, debug  Marc Wick (monte-bre.ch) –HSQL support, design debug and ideas  Duncan Smith (mightybot.com) –IP binding, security concerns, console, JMX, distributed management  Vadim Kassin (Kazakhstan Stock Exchange) –Autogenerated keys support

28. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 28 - 20/11/2003 Outline - Conclusion  Motivations  Use-Cases  C-JDBC concepts  Performance  Community  Conclusion

29. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 29 - 20/11/2003 Current status  C-JDBC 1.0b15 release –Generic JDBC 2.0 driver –Schedulers and load balancers for RAIDb 0, 1 and 2 –Fine grain query caching –JDBC recovery log –Logger/request player –Java installer –User documentation  On-going work and efforts –Octopus integration (in next release) –Horizontal scalability (for Christmas) –Dynamic reconfiguration …

30. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 30 - 20/11/2003 Prototype  C-JDBC Management Framework  Shared design

31. http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org 31 - 20/11/2003 Your questions && Your comments … http://c-jdbc.objectweb.org