Highly available web sites with Tomcat and Clustered JDBC Emmanuel Cecchet.

Highly available web sites with Tomcat and Clustered JDBC Emmanuel Cecchet

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 2 - 15/11/2004 Internet è Database tier should be  scalable  highly available  without modifying the client application  database vendor independent  on commodity hardware Motivations JDBC

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 3 - 15/11/2004 Scaling the database tier – Alternative 1 (master-slave) Internet Web frontend App. server è Cons  failover time on master failure  scalability Master

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 4 - 15/11/2004 Internet è Cons  atomic broadcast scalability  no client side load balancing  heavy modifications of the database engine Scaling the database tier – Alternative 2 (atomic broadcast) Atomic broadcast

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 5 - 15/11/2004 Scaling the database tier – Alternative 3 (SMP) Internet Web frontend App. server Well-known database vendor here Database Well-known hardware + database vendors here è Cons  Cost  Scalability limit

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 6 - 15/11/2004 Scaling the database tier – Alternative 4 (shared disks) Internet Web frontend App. server Database Disks Another well-known database vendor è Cons  still expensive hardware  availability

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 7 - 15/11/2004 Outline è RAIDb è C-JDBC è Tomcat and C-JDBC è Scalability è High availability

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 8 - 15/11/2004 RAIDb concept è Redundant Array of Inexpensive Databases è RAIDb controller  gives the view of a single database to the client  balance the load on the database backends è RAIDb levels offers various tradeoff of performance and fault tolerance

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 9 - 15/11/2004 RAIDb levels è RAIDb-0  partitioning  no duplication and no fault tolerance  at least 2 nodes

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 10 - 15/11/2004 RAIDb levels è RAIDb-1  mirroring  performance bounded by write broadcast  at least 2 nodes

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 11 - 15/11/2004 RAIDb levels è RAIDb-2  partial replication  at least 2 copies of each table for fault tolerance  at least 3 nodes

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 12 - 15/11/2004 RAIDb levels composition è RAIDb-1-0  no limit to the composition deepness

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 14 - 15/11/2004 C-JDBC overview è Middleware implementing RAIDb  100% Java implementation  open source (LGPL) è Two components  generic JDBC driver (C-JDBC driver)  C-JDBC Controller è Read-one, Write all approach  provides eager (strong) consistency è Supports heterogeneous databases

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 15 - 15/11/2004 architectural overview JVM Application server JVM C-JDBC JDBC driver C-JDBC controller JVM MySQL JDBC driver MySQL database

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 16 - 15/11/2004 Inside the C-JDBC Controller Sockets JMX

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 17 - 15/11/2004 Virtual Database è gives the view of a single database è establishes the mapping between the database name used by the application and the backend specific settings è backends can be added and removed dynamically è configured using an XML configuration file

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 18 - 15/11/2004 Authentication Manager è Matches real login/password used by the application with backend specific login/ password è Administrator login to manage the virtual database

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 19 - 15/11/2004 Scheduler è Manages concurrency control è Specific implementations for RAIDb 0, 1 and 2 è Pass-through è Optimistic and pessimistic transaction level  uses the database schema that is automatically fetched from backends

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 20 - 15/11/2004 Request cache è 3 optional caches  tunable sizes è parsing cache  parse request skeleton only once  INSERT INTO t VALUES (?,?,?) è metadata cache  column metadata  fields of a request è result cache  caches results from SQL requests  tunable consistency  fine grain invalidations  optimizations for findByPk requests

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 21 - 15/11/2004 Load balancer 1/2 è RAIDb-0  query directed to the backend having the needed tables è RAIDb-1  read executed by current thread  write executed in parallel by a dedicated thread per backend  result returned if one, majority or all commit  if one node fails but others succeed, failing node is disabled è RAIDb-2  same as RAIDb-1 except that writes are sent only to nodes owning the updated table

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 22 - 15/11/2004 Load balancer 2/2 è Static load balancing policies  Round-Robin (RR)  Weighted Round-Robin (WRR) è Least Pending Requests First (LPRF)  request sent to the node that has the shortest pending request queue  efficient even if backends do not have homogeneous performance

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 23 - 15/11/2004 Connection Manager è C-JDBC driver provides transparent connection pooling è Connection pooling for a backend  no pooling  blocking pool  non-blocking pool  dynamic pool è Connection pools defined on a per login basis  resource management per login  dedicated connections for admin

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 24 - 15/11/2004 Recovery Log è Checkpoints are associated with database dumps è Record all updates and transaction markers since a checkpoint è Used to resynchronize a database from a checkpoint è JDBCRecoveryLog  store log information in a database  can be re-injected in a C-JDBC cluster for fault tolerance

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 25 - 15/11/2004 Functional overview (read) connect myDB connect login, password execute SELECT * FROM t

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 26 - 15/11/2004 Functional overview (write) execute INSERT INTO t …

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 27 - 15/11/2004 Failures è No 2 phase- commit  parallel transactions  failed nodes are automatically disabled execute INSERT INTO t …

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 29 - 15/11/2004 Internet è Apache clustering  L4 switch, RR-DNS, One-IP techniques, LVS, … è Tomcat clustering  mod_jk (T4), mod_proxy/mod_rewrite (T5), session replication è Database clustering  C-JDBC Highly available web site mod-jk RR-DNS Parsing cache Result cache Metadata cache

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 30 - 15/11/2004 Result cache è Cache contains a list of SQL→ResultSet è Policy defined by queryPattern→Policy è 3 policies  EagerCaching: variable granularities for invalidations  RelaxedCaching: invalidations based on timeout  NoCaching: never cached RUBiS bidding mix with 450 clients No cacheCoherent cache Relaxed cache Throughput (rq/min)389241844215 Avg response time801 ms284 ms134 ms Database CPU load100%85%20% C-JDBC CPU load-15%7%

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 31 - 15/11/2004 Internet è Multiple databases  choosing RAIDb level  recovery log for adding nodes dynamically recovering from failures Highly available web site

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 32 - 15/11/2004 Configuring Tomcat 5 è Same setting for all Tomcat nodes  Copy c-jdbc-driver.jar to $CATALINA_HOME/common/lib è Without connection pooling  1. load driver Class.forName("org.objectweb.cjdbc.driver.Driver").newInstance();  2. get a connection Connection conn = DriverManager. getConnection("jdbc:cjdbc://host/myDB","user","password");

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 33 - 15/11/2004 Configuring Tomcat 5 è With connection pooling factory org.apache.commons.dbcp.BasicDataSourceFactory driverClassName org.objectweb.cjdbc.driver.Driver url jdbc:cjdbc://localhost/myDB

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 34 - 15/11/2004 Configuring C-JDBC (1/3) è Virtual database configuration file

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 35 - 15/11/2004 Configuring C-JDBC (2/3) <DatabaseBackend name="node1“ driver="org.gjt.mm.mysql.Driver“ url="jdbc:mysql://host1/backend1“ connectionTestStatement="select 1">

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 36 - 15/11/2004 Configuring C-JDBC (3/3)

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 37 - 15/11/2004 Internet … Controller replication

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 39 - 15/11/2004 C-JDBC vertical scalability è allows nested RAIDb levels è allows tree architecture for scalable write broadcast è necessary with large number of backends è C-JDBC driver re-injected in C-JDBC controller

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 40 - 15/11/2004 C-JDBC vertical scalability è RAIDb-1-1 with C-JDBC è no limit to composition deepness

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 41 - 15/11/2004 Vertical scalability è Addresses JVM scalability issues è Distributing large number of connections on many backends

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 42 - 15/11/2004 TPC-W benchmark (Amazon.com) è Nearly linear speedups with the shopping mix

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 44 - 15/11/2004 Controller replication è Prevent the controller from being a single point of failure è Group communication for controller synchronization è C-JDBC driver supports multiple controllers with automatic failover jdbc:c-jdbc://node1:25322,node2:12345/myDB

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 45 - 15/11/2004 Controller replication jdbc:cjdbc://node1,node2/myDB Total order reliable multicast

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 46 - 15/11/2004 Mixing horizontal & vertical scalability

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 47 - 15/11/2004 Building initial checkpoint è Octopus is an ETL tool è Use Octopus to store a dump of the initial database state è Currently done by the user using the database specific dump tool

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 48 - 15/11/2004 Logging è Backend is enabled è All database updates are logged (SQL statement, user, transaction, …)

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 49 - 15/11/2004 Adding new backends 1/3 è Add new backends while system online è Restore dump corresponding to initial checkpoint with Octopus

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 50 - 15/11/2004 Adding new backends 2/3 è Replay updates from the log

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 51 - 15/11/2004 Adding new backends 3/3 è Enable backends when done

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 52 - 15/11/2004 Making new checkpoints (1/3) è Disable one backend to have a coherent snapshot è Mark the new checkpoint entry in the log è Use Octopus to store the dump

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 53 - 15/11/2004 Making new checkpoints (2/3) è Replay missing updates from log

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 54 - 15/11/2004 Making new checkpoints (3/3) è Re-enable backend when done

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 55 - 15/11/2004 Handling failures è A node fails! è Automatically disabled but should be fixed or changed by administrator

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 56 - 15/11/2004 Recovery 1/3 è Restore latest dump with Octopus

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 57 - 15/11/2004 Recovery 2/3 è Replay missing updates from log

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 58 - 15/11/2004 Recovery 3/3 è Re-enable backend when done

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 59 - 15/11/2004 Fault tolerant recovery log UPDATE statement

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 60 - 15/11/2004 Demo – xPetstore/HSQL backend1 HypersonicSQL in-memory database xPetstore Servlet è http://xpetstore.sourceforge.net/  open source implementation of Petstore  Servlet solution: Velocity, WebWork, Sitemesh, POJO and Hibernate.

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 61 - 15/11/2004 Demo 2 – xPetstore/C-JDBC/HSQL backend1 backend2 HypersonicSQL in-memory databases JMX Recovery log RAIDb-1 xpetstore Servlet C-JDBC administration console

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 62 - 15/11/2004 C-JDBC today è Web site  ~200.000 hits/month  ~ 26.000 downloads  EU (18 countries) 36%, US 28%, Japan 12%, China 5%, Canada 4%, Australia 4%, India 3%, Brazil 2%, … è Community  27 committers both industrial & academics  c-jdbc@objectweb.org: >200 subscribers, 200-300 msgs/month  translation in japanese, italian, chinese, turkish, french è RPM on JPackage.org

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 63 - 15/11/2004 What’s next? è Tribe (.objectweb.org)  replacement for JGroups  uniform total order broadcast optimized for clusters è LeWYS (.objectweb.org)  hardware and software monitoring  monitoring repository è Distributed query execution è X509 certificates è Optimized support for edge-side servers and interconnected clusters

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 64 - 15/11/2004 Conclusion è RAIDb  RAID-like scheme for databases è C-JDBC  open source middleware for database replication  performance scalability  high availability è Tomcat & C-JDBC  no application modification required  RDBMS vendor independent

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 65 - 15/11/2004 Q&A _________ Thanks to all users and contributors... http://c-jdbc.objectweb.org

Bonus slides

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 67 - 15/11/2004 Current limitations è JDBC only è Distributed joins è Updatable ResultSets è XA support through XAPool only è transparent controller failover not supported when using horizontal scalability with JGroups è network partition/reconciliation not supported

HORIZONTAL SCALABILITY

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 69 - 15/11/2004 Horizontal scalability è JGroups for controller synchronization è Groups messages for writes only

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 70 - 15/11/2004 Horizontal scalability è Centralized write approach issues è Issues with transactions assigned to connections

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 71 - 15/11/2004 Horizontal scalability è General case for a write query  3 multicast + 2n unicast

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 72 - 15/11/2004 Horizontal scalability è Solution: No backend sharing  1 multicast + n unicast [+ 1 multicast]

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 73 - 15/11/2004 Horizontal scalability è Issues with JGroups  resources needed by a channel  instability of throughput with UDP  performance scalability è TCP better than UDP but  unable to disable reliability on top of TCP  unable to disable garbage collection  ordering implementation is sub-optimal è Need for a new group communication layer optimized for cluster

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 74 - 15/11/2004 Horizontal scalability è JGroups performance on UDP/FastEthernet

USE CASES

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 76 - 15/11/2004 Budget High Availability è High availability infrastructure “on a budget” è Typical eCommerce setup è http://www.budget-ha.com

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 77 - 15/11/2004 OpenUSS: University Support System è eLearning è High availability è Portability  Linux, HP-UX, Windows  InterBase, Firebird, PostgreSQL, HypersonicSQL è http://openuss.sourceforge.net

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 78 - 15/11/2004 Flood alert system è Disaster recovery è Independent nodes synchronized with C-JDBC è VPN for security issues è http://floodalert.org

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 79 - 15/11/2004 J2EE benchmarking è Large scale J2EE clusters è http://jmob.objectweb.org Interne t emulated users

PERFORMANCE

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 81 - 15/11/2004 TPC-W è Browsing mix performance

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 82 - 15/11/2004 TPC-W è Shopping mix performance

http://c-jdbc.objectweb.org/ - c-jdbc@objectweb.org ApacheCon’04 - 83 - 15/11/2004 TPC-W è Ordering mix performance

Highly available web sites with Tomcat and Clustered JDBC Emmanuel Cecchet.

Similar presentations

Presentation on theme: "Highly available web sites with Tomcat and Clustered JDBC Emmanuel Cecchet."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Highly available web sites with Tomcat and Clustered JDBC Emmanuel Cecchet.

Similar presentations

Presentation on theme: "Highly available web sites with Tomcat and Clustered JDBC Emmanuel Cecchet."— Presentation transcript:

Similar presentations

About project

Feedback