Event Processing A Perspective From Oracle Dieter Gawlick, Shailendra Mishra Oracle Corporation March,

2 Event Processing Symposium Agenda Elements Support for event processing in Oracle’s products Some unique contributions by Oracle Which application are best served? Additional important applications Major trends Major challenges Questions to the research community

3 March, Event Processing Symposium Elements Publication (Publishers) Dissemination (Disseminators) Consumption (Consumers) publishes messages delivers messages Applications Files/DBs FeedsSensor DataPropagations

4 March, Event Processing Symposium Elements Publication (Publishers) Dissemination (Disseminators) Consumption (Consumers) publishes messages delivers messages Messages Reports about Events State Changes may generate may generate Event generation: * Procedural - no external specifications * Declaratives - In reaction to external specifications

5 March, Event Processing Symposium Elements Publication (Publishers) Dissemination (Disseminators) Consumption (Consumers) publishes messages delivers messages The Core Element Stream Single or multiple steps Local or distributed Message Management Select Process Compose Retain Deliver/propagate

6 March, Event Processing Symposium Elements Publication (Publishers) Dissemination (Disseminators) Consumption (Consumers) publishes messages delivers messages Applications Files/DBs FeedsActuatorsPropagations

7 March, Event Processing Symposium Some Notes Publishers create solicited and unsolicited (spam) messages – Solicited messages are based on subscriptions* – Subscriptions may be against messages, events, and state transitions (Heisenberg events) – Consumers can also act as publishers – creates value chains Dissemination moves messages from publishers to consumers – Dissemination may be a multi-step, distributed process – Selection and composition can access single and multiple streams – There is rich support for aggregation (e.g., running average) and user functions Everything is driven by meta-data – Meta-data describe publishers, consumers, dissemination and data structures (for states/events/messages) CEP/Streams processing is focused on management of event data and as such may be considered as a subset of dissemination * (Continuous) Queries can act as subscriptions

8 March, Event Processing Symposium Support for Event Processing in Oracle’s Products Applications features – Business Events – a large number of application triggers Application Server features – Sync/Async Messaging – Event Capture – Event Correlation – Metrics & KPIs – Real time dashboards – Alerts Database features – Triggers – Streams/AQ – Rules support – Asynchronous client notification – Data Change Notification

9 March, Event Processing Symposium Some Unique Contributions by Oracle JMS – Leveraging and extending Oracle stack for messaging resulting in improved functionality and operational characteristics – Support for Abstract data-types over and above standard JMS types – Full support for transactions, recovery and high availability BPEL/BAM/RFID/Alerts – Uniform declarative framework to do event processing  Every node is data stream processor that speaks SQL-ese  Hierarchical, stream based views as an organizing principle  Allows system to adapt over time  Optimization across multiple applications  Simplified programming constructs

10 March, Event Processing Symposium Some Unique Contributions by Oracle Streams/AQ – Extension of database to provide a messaging solution with improved functionality and operational characteristics e.g., auditing and tracking, security, reliability and availability – Declarative specification of publication, dissemination and consumption – Recoverability with minimal journaling Rules technology – Rules technology evolved as key data base technology – Support for SQL, XML, Text and spatial data – Support of a very large number (>10**6) of rules with moderate rate of change, management of rule sets, auditing of rules and data – Very low cost of evaluation for data materialized in data bases – Expression as data, provides a new way to manage and process subscriptions – Much of the rules technology is shared with query processing – a subscription is a query on future data

11 March, Event Processing Symposium Which Applications are Best Served? Messaging for mission critical applications – e.g., exchanges, fulfillment – Extended functionality:  Extensive type support, auditing, dissemination and more  Reduced development effort – ease of use – Improved operational characteristics:  High performance, scalability, reliability, security or more Replication – Declarative publication, dissemination and consumption – High reliability with minimal journaling Event notification – Declarative, automatic event creation in response to database changes - subscribe to publish – Sophisticated dissemination

12 March, Event Processing Symposium Which Applications are Best Served? Complex event processing – e.g., management and processing of sensor data – Events from many source types – Declarative event composition and non events – Auditing and tracking (data and rules) – Management of rule sets with user specified policies – Mutual filtering – Metrics and KPI – Alerts – Real Time dashboards

13 March, Event Processing Symposium Which Applications are Next? Security Analytics – Intrusion Detection – Security log correlation for damage assessment XML Streams – Correlating complex XML streams – Detecting patterns in XML streams without staging. RSS – Declarative creation and/or consumption of RSS Streams – data bases as publishers/consumers of choice – (Fine Grin) Security, mutual filtering – Push support Information Driven Architecture – e.g., SensorNet, VIRT – Event specification and optionally dissemination and consumption are part of the data specification – Event specification and processing reflect the rules and interest of (internal and external) communities

14 March, Event Processing Symposium Major Trends* Moore’s law implies a new class of computers about every 10 years – The next step enables the unambiguous use of sensors (and actuators) We are on the verge of a vast of improvement in the ‘Spatial and temporal fidelity at which We instrument and analyze the physical world’ Resulting in an avalanche of data We are moving towards proactive and probabilistic computing – Computers will provide us with Valuable Information at the Right Time based on Conditions Of Interest * This slide is based on ideas documented by Hans Mulder (Intel) and Rick Hayes-Roth (NPS)

15 March, Event Processing Symposium

16 March, Event Processing Symposium The Helicopter’s Filtered World Model Flight Planned Route

17 March, Event Processing Symposium Major Challenges Agree on and document a small number of use cases (3-5) – Capture functional and operational requirements – Capture desired (user) development support – Capture cost constraints for operations (IT and end user) Create a Terminology Identify and characterize missing standards; e.g., – Continuous queries (for streams and states) – Dissemination – a generic model – Specification of publisher/consumer communities Specify benchmark

18 March, Event Processing Symposium Questions to the Research Community There is a need for an extensible spatial and temporal view of the world – What are major organizing principals? There is a need for data models and CQ languages for streams and state transitions – How can events (and messages) be derived from state transitions? – How can dissemination and community models be integrated with CQ’s? – How does one find relevant continuous queries? Rules technology is a core element of event processing – What is the state of the art for which patterns? – Which fundamental challenges exist? Dissemination is another core element of event processing – Is there a generic model and what is it? Why do researchers pay so little attention to evolving technologies in existing products?

19 March, Event Processing Symposium A Q & Q U E S T I O N S A N S W E R S