Presentation is loading. Please wait.

Presentation is loading. Please wait.

Sun Proprietary/Confidential Information © Sun Microsystems 2005 Intelligent Event Processing (IEP) Aggregation Partition Filtering Correlation.

Similar presentations


Presentation on theme: "Sun Proprietary/Confidential Information © Sun Microsystems 2005 Intelligent Event Processing (IEP) Aggregation Partition Filtering Correlation."— Presentation transcript:

1 Sun Proprietary/Confidential Information © Sun Microsystems 2005 Intelligent Event Processing (IEP) Aggregation Partition Filtering Correlation

2 Sun Proprietary/Confidential Information © Sun Microsystems 2005 Aggregation Time-based Aggregator Window Aggregator Tuple-based Aggregator

3 Sun Proprietary/Confidential Information © Sun Microsystems 2005 Time-based Aggregator Compute statistical summery over a fixed time period. Statistics: sum, average, median, lower-quantile, upper- quantile, min, max, standard-deviation, sum-of-square, outlier-percentage. Example: Given a stream of transactions of a stock, computes a new stream that holds hourly min, average, and max of the stock price.

4 Sun Proprietary/Confidential Information © Sun Microsystems 2005 Tuple-based Aggregator Compute statistical summery over a fixed number of events. Statistics: same as time-based aggregator. Example: Given a stream of transactions of a stock, computes a new stream that holds the min, average, and max of the stock price of every 10 transactions.

5 Sun Proprietary/Confidential Information © Sun Microsystems 2005 Window Aggregator Compute statistical summery over a sliding window Statistics: same as time-based aggregator. Example: Given the latest 2 hour window of transactions of a stock, computes the min, average, and max of the stock price of those transactions. As time passes by, the statistical summery will change too.

6 Sun Proprietary/Confidential Information © Sun Microsystems 2005 Filtering Standard Filtering Filtering by Hypothesis Testing Filtering by Point Estimation

7 Sun Proprietary/Confidential Information © Sun Microsystems 2005 Standard Filtering Example: Given a stream of transactions of a stock, computes a new stream that holds only those transactions of which the stock price is > $50.

8 Sun Proprietary/Confidential Information © Sun Microsystems 2005 Filtering by Point Estimation Analyze historical data, and build a Statistical model. Create a filter that filters out only those events that are considered outliers by the statistical model. Example: Given enough historical measurements of human body: (weight (kg), height (cm)), one build a Statistical model: weight ~ -105 + height + N(0, 10). Given a stream of measurement data, computes a new stream that holds those measurement of which weight is outside of (-105 + height - 20, -105 + height + 20)

9 Sun Proprietary/Confidential Information © Sun Microsystems 2005 Filtering by Hypothesis Testing Analyze historical data and make up a hypothesis. Creates a filter that filters out only those groups of events that failed the hypothesis. Example: Given enough historical measurements of products produced by an assembly line, one found that the defect rate is 2%. Make up a hypothesis: defect rate <= 2% Given a stream of measurement data, computes a new stream that holds alert events. Test the hypothesis for every 100 measurements. If the defect rate is Statistically greater than 2%, generate an alert event.

10 Sun Proprietary/Confidential Information © Sun Microsystems 2005 Partition Standard Partition Partition by Classification

11 Sun Proprietary/Confidential Information © Sun Microsystems 2005 Standard Partition Example: Given a stream of stock transactions, computes a window that holds the latest 10 transactions for each stock symbol.

12 Sun Proprietary/Confidential Information © Sun Microsystems 2005 Partition by Classification Analyze historical data and build a Statistical classification model. Creates a partition window that holds the latest N events for each classification group.

13 Sun Proprietary/Confidential Information © Sun Microsystems 2005 Correlation Relational Join of Multiple Windows

14 Sun Proprietary/Confidential Information © Sun Microsystems 2005 Relational Join of Multiple Windows Example: Given the latest 2 hour window of stock transactions, and the latest 2 hour window of news, computes the latest 2 hour window of insider-trades by joining the trader’s name with the name mentioned in the news.


Download ppt "Sun Proprietary/Confidential Information © Sun Microsystems 2005 Intelligent Event Processing (IEP) Aggregation Partition Filtering Correlation."

Similar presentations


Ads by Google