Download presentation
Presentation is loading. Please wait.
Published byLouisa Nelson Modified over 9 years ago
1
Sun Proprietary/Confidential Information © Sun Microsystems 2005 Intelligent Event Processing (IEP) Aggregation Partition Filtering Correlation
2
Sun Proprietary/Confidential Information © Sun Microsystems 2005 Aggregation Time-based Aggregator Window Aggregator Tuple-based Aggregator
3
Sun Proprietary/Confidential Information © Sun Microsystems 2005 Time-based Aggregator Compute statistical summery over a fixed time period. Statistics: sum, average, median, lower-quantile, upper- quantile, min, max, standard-deviation, sum-of-square, outlier-percentage. Example: Given a stream of transactions of a stock, computes a new stream that holds hourly min, average, and max of the stock price.
4
Sun Proprietary/Confidential Information © Sun Microsystems 2005 Tuple-based Aggregator Compute statistical summery over a fixed number of events. Statistics: same as time-based aggregator. Example: Given a stream of transactions of a stock, computes a new stream that holds the min, average, and max of the stock price of every 10 transactions.
5
Sun Proprietary/Confidential Information © Sun Microsystems 2005 Window Aggregator Compute statistical summery over a sliding window Statistics: same as time-based aggregator. Example: Given the latest 2 hour window of transactions of a stock, computes the min, average, and max of the stock price of those transactions. As time passes by, the statistical summery will change too.
6
Sun Proprietary/Confidential Information © Sun Microsystems 2005 Filtering Standard Filtering Filtering by Hypothesis Testing Filtering by Point Estimation
7
Sun Proprietary/Confidential Information © Sun Microsystems 2005 Standard Filtering Example: Given a stream of transactions of a stock, computes a new stream that holds only those transactions of which the stock price is > $50.
8
Sun Proprietary/Confidential Information © Sun Microsystems 2005 Filtering by Point Estimation Analyze historical data, and build a Statistical model. Create a filter that filters out only those events that are considered outliers by the statistical model. Example: Given enough historical measurements of human body: (weight (kg), height (cm)), one build a Statistical model: weight ~ -105 + height + N(0, 10). Given a stream of measurement data, computes a new stream that holds those measurement of which weight is outside of (-105 + height - 20, -105 + height + 20)
9
Sun Proprietary/Confidential Information © Sun Microsystems 2005 Filtering by Hypothesis Testing Analyze historical data and make up a hypothesis. Creates a filter that filters out only those groups of events that failed the hypothesis. Example: Given enough historical measurements of products produced by an assembly line, one found that the defect rate is 2%. Make up a hypothesis: defect rate <= 2% Given a stream of measurement data, computes a new stream that holds alert events. Test the hypothesis for every 100 measurements. If the defect rate is Statistically greater than 2%, generate an alert event.
10
Sun Proprietary/Confidential Information © Sun Microsystems 2005 Partition Standard Partition Partition by Classification
11
Sun Proprietary/Confidential Information © Sun Microsystems 2005 Standard Partition Example: Given a stream of stock transactions, computes a window that holds the latest 10 transactions for each stock symbol.
12
Sun Proprietary/Confidential Information © Sun Microsystems 2005 Partition by Classification Analyze historical data and build a Statistical classification model. Creates a partition window that holds the latest N events for each classification group.
13
Sun Proprietary/Confidential Information © Sun Microsystems 2005 Correlation Relational Join of Multiple Windows
14
Sun Proprietary/Confidential Information © Sun Microsystems 2005 Relational Join of Multiple Windows Example: Given the latest 2 hour window of stock transactions, and the latest 2 hour window of news, computes the latest 2 hour window of insider-trades by joining the trader’s name with the name mentioned in the news.
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.