Data Grid Patterns Brian Oliver | Global Solutions Architect | Oracle Corporation | JBFOne 2008

Agenda Introduction to Data Grids Introduction to Oracle Coherence Introduction to Data Grid Patterns The Command Pattern The Functor Pattern Vorstellung der Thematik Distributed Cache Funktionsweise anhand von Coherence wird erklärt Die Ideen bezüglich Distributed Cache werden vorgestellt: Ikesa cache Verwendung Grid anstatt AQ Die Servicewolke Demo The Store and Forward Messaging Pattern

Agenda Introduction to Data Grids Introduction to Oracle Coherence Introduction to Data Grid Patterns The Command Pattern The Functor Pattern Vorstellung der Thematik Distributed Cache Funktionsweise anhand von Coherence wird erklärt Die Ideen bezüglich Distributed Cache werden vorgestellt: Ikesa cache Verwendung Grid anstatt AQ Die Servicewolke Demo The Store and Forward Messaging Pattern

Introduction to Data Grids A Data Grid is a data management system for application objects Application objects are managed across multiple servers (typically stored in memory) Essential for Applications that require low response time very high throughput predictable scalability continuous availability information reliability

Introduction to Data Grids Data management system Is it a Container? Can it be Embedded? Application objects What languages? Not necessarily relational! Query support? Shared across multiple servers Architecture? How are the servers coordinated? How are the servers organized? Low response time Seconds? Milliseconds? Microseconds?

Introduction to Data Grids Very high throughput? Transactions or Operations Per Second? MB per Second? Predictable scalability Accurately predict future resources based on current utilization Continuous availability During failure, recovery, scale-out, upgrades? Information reliability Consistency? Isolation? Durability? Persistence?

Agenda Introduction to Data Grids Introduction to Oracle Coherence Introduction to Data Grid Patterns The Command Pattern The Functor Pattern Vorstellung der Thematik Distributed Cache Funktionsweise anhand von Coherence wird erklärt Die Ideen bezüglich Distributed Cache werden vorgestellt: Ikesa cache Verwendung Grid anstatt AQ Die Servicewolke Demo The Store and Forward Messaging Pattern

Introduction to Oracle Coherence Peer-to-Peer Clustering No Single Points of Failure No Single Points of Bottleneck No Masters / Slaves / Registries etc All members have responsibility to; Manage Cluster Health & Data Perform Processing and Queries Work as a “team” in parallel Communication is point-to-point (not TCP/IP) and/or one-to-many Scale to limit of the back-plane Use with commodity infrastructure Linearly Scalable By Design

Introduction to Oracle Coherence Data is automatically partitioned and load-balanced across the Server Cluster Data is synchronously replicated for continuous availability Servers monitor the health of each other When in doubt, servers work together to diagnose status Healthy servers assume responsibility for failed server (in parallel) Continuous Operation: No interruption to service or data loss due to a server failure

Introduction to Oracle Coherence Dynamically scale-out during operation Data automatically load-balanced to new servers in the cluster No repartitioning required No reconfiguration required No interruption to service during scale-out Scale capacity and processing on-the-fly

Introduction to Oracle Coherence Software Development Library Pure Java 1.4.2+ (data grid) Pure .Net 1.1 and 2.0 (client) Pure C++ (client) No Third-Party Dependencies No Open Source Dependencies Proprietary Network Stack (Peer-To-Peer model) Other Libraries Support… Database and File System Integration Top Link and Hibernate Http Session Management

Agenda Introduction to Data Grids Introduction to Oracle Coherence Introduction to Data Grid Patterns The Command Pattern The Functor Pattern Vorstellung der Thematik Distributed Cache Funktionsweise anhand von Coherence wird erklärt Die Ideen bezüglich Distributed Cache werden vorgestellt: Ikesa cache Verwendung Grid anstatt AQ Die Servicewolke Demo The Store and Forward Messaging Pattern

Introduction to Data Grid Patterns Implementations of traditional patterns, but on a Data Grid! Inter-process Architectural Patterns The Command Pattern The Functor Pattern The Task Executor Pattern Intra-process Architectural Patterns Store and Forward Messaging Push Replication The Coherence Incubator http://coherence.oracle.com/display/INCUBATOR Example Implementations, Source Code, Documentation

Agenda Introduction to Data Grids Introduction to Oracle Coherence Introduction to Data Grid Patterns The Command Pattern The Functor Pattern Vorstellung der Thematik Distributed Cache Funktionsweise anhand von Coherence wird erklärt Die Ideen bezüglich Distributed Cache werden vorgestellt: Ikesa cache Verwendung Grid anstatt AQ Die Servicewolke Demo The Store and Forward Messaging Pattern

The Command Pattern: Definition An action to be performed zero or more times at some point in the future should be represented as an object called a Command All necessary parameters required to perform the said action, should be encapsulated as attributes with in the said Command object The Command object must provide a mechanism to perform the action (typically represented an execute method defined on the said Command object)

The Command Pattern: Extended Semantics (for Data Grids) A Command may only be executed within the scope of a target object, called a Context To execute a Command it must be submitted to a Context Once submitted, Commands are executed asynchronously Commands are executed one-at-a-time, in the order in which they are submitted A Command may not return a value to the application that submitted the said Command (for this use the Functor Pattern)

Command Pattern Demonstration

The Command Pattern: How it works! Step 1: The Initial Environment

The Command Pattern: How it works! Step 2: Creating a Context

The Command Pattern: How it works! Step 3: Registering a Context

The Command Pattern: How it works The Command Pattern: How it works! Step 4: Command Executor Established (automatic)

The Command Pattern: How it works! Step 5: Creating a Command

The Command Pattern: How it works! Step 6: Submitting a Command

The Command Pattern: How it works The Command Pattern: How it works! Step 7: Submitted Commands are Queued (automatic)

The Command Pattern: How it works The Command Pattern: How it works! Step 8: Submitted Commands are Executed (automatic)

The Command Pattern: How it works The Command Pattern: How it works! Step 9: Executed Commands are Removed (automatic)

Agenda Introduction to Data Grids Introduction to Oracle Coherence Introduction to Data Grid Patterns The Command Pattern The Functor Pattern Vorstellung der Thematik Distributed Cache Funktionsweise anhand von Coherence wird erklärt Die Ideen bezüglich Distributed Cache werden vorgestellt: Ikesa cache Verwendung Grid anstatt AQ Die Servicewolke Demo The Store and Forward Messaging Pattern

The Functor Pattern: Definition Just like the Command Pattern … An action to be performed zero or more times at some point in the future should be represented as an object called a Functor All necessary parameters required to perform the said action, should be encapsulated as attributes within the said Functor object The Functor object must provide a mechanism to perform the action (typically represented an execute method defined on the said Functor object) The result of a Functor execution is returned to the submitter using a Java 5+ Future<?>

Functor Pattern Demonstration

The Functor Pattern: Definition Just like the Command Pattern … An action to be performed zero or more times at some point in the future should be represented as an object called a Functor All necessary parameters required to perform the said action, should be encapsulated as attributes with in the said Functor object The Functor object must provide a mechanism to perform the action (typically represented an execute method defined on the said Functor object) Additional Requirement The result of a Functor execution is returned to the submitter using a Java 5+ Future<?>

Agenda Introduction to Data Grids Introduction to Oracle Coherence Introduction to Data Grid Patterns The Command Pattern The Functor Pattern Vorstellung der Thematik Distributed Cache Funktionsweise anhand von Coherence wird erklärt Die Ideen bezüglich Distributed Cache werden vorgestellt: Ikesa cache Verwendung Grid anstatt AQ Die Servicewolke Demo The Store and Forward Messaging Pattern

The Messaging Pattern: Definition Payload, typically represented as a Message object, may be sent to a Destination from a Sender (also commonly known as a Publisher) It is the responsibility of the infrastructure managing the Destination to ensure that Messages (arriving at the said Destination) are then stored (in some manner) and consequently forwarded (in the order in which they were sent) to one or more Receivers (also commonly known as Subscribers)

The Messaging Pattern: Definition Subscribers consume (receive and acknowledge receipt) Messages from the Destination in the order in which they were sent from the Publishers The infrastructure managing the Messages appropriately cleans-up (remove and garbage collect) the Messages that have been consumed by Subscribers The type of the Destination determines the method of delivery to the Subscribers on that Destination

The Messaging Pattern: Definition A Topic Destination (or Topic) will store and forward Messages to all of the Subscribers of the said Topic Destination This form of Message delivery is often called "publish-and-subscribe messaging", "one- to-many messaging" or "the observer pattern" A Queue Destination (or Queue) will store and forward Messages to at most one of the Subscribers of the said Queue Destination For each Message a different Subscriber may be used, but this is implementation and runtime dependent. This form of Message delivery is often called "point-to-point messaging" or "one-to-one messaging"

The Messaging Pattern: Definition A Message may be Persistent or Non-Persistent In the case of Persistent Messages, the infrastructure managing the Destination must safely store the said Messages to a persistent (and recoverable) storage device so that in the case of infrastructure failure, Messages may be recovered (not lost) A Subscriber to a Topic is either Durable or Non-Durable Durable Subscriptions allow the system implementing the Subscriber to terminate and return without losing Messages that may have been delivered during the outage (or disconnection)

Messaging Pattern Demonstration

Conclusions Data Grids provide more than just “caching” capabilities Events Queries Processing Analytics Data Grids (like Coherence) provide reliable (but transaction-free) infrastructure for implementing all kinds of advanced architectural patterns Data Grids help solve the scalability and performance challenges we have traditionally faced, using pure Java

Ihr IT-Partner Thanks!