Download presentation
Presentation is loading. Please wait.
Published byCuthbert Foster Modified over 8 years ago
3
Comprehensive Flexible Global Storage and Search Responsive Available Secure Manageable Federation Coordination Consolidation Transformation Synchronization Composition Monitor Analyze Plan Collaborate Insight On client On premise Partner-hosted Microsoft-hosted Delivery Productive Interoperable Supports key standards Data Programmability
4
Database Engine Relational Data Management Replication Full Text Search Integration Services ETL Processing Data Profiling StreamInsight* Complex Event Processing Analysis Services Classic OLAP Data Management Data Mining PowerPivot* Self Service Analytics Reporting Services Managed Reporting Self Service Reporting Embedded Reporting Master Data Services* Master Data Management
7
New application scenario for SQL Server Process high volumes of event stream data with low latency Compliments existing solutions Complex Event Processing (CEP) Integrated development platform for CEP applications Modular, provider-based architecture Input adapters read event streams Event models support diverse patterns of events Runtime engine hosts standing LINQ queries built in.NET Output adapters transmit results Introducing StreamInsight
10
DatabaseCEP QueriesAd-hoc on stored dataContinuous standing queries LatencySecondsMilliseconds Data RateHundreds per SecondTens of Thousands per Second request response Event output stream input stream
12
Relational Database Applications Financial Trading Applications Aggregate Data Rate (Events/sec) 010100100010000100000higher Months Days Hours Minutes Seconds 100 ms < 1 ms Operational Analytics Applications, e.g., Logistics, etc. Manufacturing Applications Monitoring Applications CEP Target Scenarios Data Warehousing Applications Web Analytics Applications
23
Progress – ApamaIBM – Open ESB West Global – Total InsightCoral8 Bristol Technology – SenActiveGmbH – Real Time Monitoring StreambaseAgent Logic InetcoTibco Oracle – Fusion MiddlewareEvent Zero SynderaRulecare
30
Data Sources, Operations, Assets, Feeds, Sensors, Devices Monitor & Record Monitor & Record Operational Data Store & Archive CEP Engine f(x) g(y) CEP Engine f(x) f'(x) g(y) h(x,y) History Deploy Results f'(x) h(x,y) Manage & Benefit Manage & Benefit Mine & Design Mine & Design Input Data Streams Output Data Streams
31
StreamInsight Architecture
32
Data Sources Aggregation & Correlation CEP CEP for lightweight processing and filtering CEP for aggregation and correlation of in-flight events CEP for complex analytics including historical data Devices Sensors Web servers Feeds CEP Complex Analytics & Mining
41
var inputConf = new TextFileInputConfig("MyData.csv"); var outputConf = new TextFileOutputConfig("Result.csv"); var inputStream = CepStream.Create("inputStream", typeof(TextFileInputFactory),inputConf, EventShape.Point); CepStream filter = from e in inputStream where e.value > 10 select e; Query query = filter.ToQuery(typeof(TextFileOutputFactory), outputConf, EventShape.Interval, StreamEventOrder.FullyOrdered); query.Start(); // perform work or wait for output adapter to signal complete query.Stop();
44
Server server = Server.Create(); Application application = server.CreateApplication("app1"); InputAdapter ia = application.CreateInputAdapter ( "InputAdapterName", "Some description"); OutputAdapter oa = application.CreateOutputAdapter ( "OutputAdapterName", "Some other description"); CepStream inputStream = CepStream.Create("inputStream"); var highWaterMarkAlert = from e in inputStream where e.value > 99 select e; Server server = Server.Create(); Application application = server.CreateApplication("app1"); InputAdapter ia = application.CreateInputAdapter ( "InputAdapterName", "Some description"); OutputAdapter oa = application.CreateOutputAdapter ( "OutputAdapterName", "Some other description"); CepStream inputStream = CepStream.Create("inputStream"); var highWaterMarkAlert = from e in inputStream where e.value > 99 select e; Server server = Server.Create(); Application application = server.CreateApplication("app1"); InputAdapter ia = application.CreateInputAdapter ( "InputAdapterName", "Some description"); OutputAdapter oa = application.CreateOutputAdapter ( "OutputAdapterName", "Some other description"); CepStream inputStream = CepStream.Create("inputStream"); var highWaterMarkAlert = from e in inputStream where e.value > 99 select e;
45
QueryTemplate qt1 = application.CreateQueryTemplate( "highWaterMarkAlert", highWaterMarkAlert); QueryBinder qb = new QueryBinder(qt1); qb.BindProducer("inputStream", ia, new MyInputAdapterConfig(), EventShape.Interval); qb.AddConsumer("queryresult", oa, new MyOutputAdapterConfig(), EventShape.Point, StreamEventOrder.FullyOrdered); Query q1 = application.CreateQuery( "query1", qb, "Query Description"); q1.Start();
46
// Connect to a remote CEP server. Server server = Server.Connect( new System.ServiceModel.EndpointAddress( @"http://localhost:12345/CEP")); // Retrieve the application app1 from the server. Application application = server.Applications["app1"]; // Retrieve existing query template, qt1, from server. QueryTemplate qt1 = application.QueryTemplates["qt1"];
49
// Connect to a remote CEP server. Server server = Server.Connect( new System.ServiceModel.EndpointAddress( @"http://localhost:12345/CEP")); // Retrieve the application app1 from the server. Application application = server.Applications["app1"]; // Retrieve existing query template, qt1, from server. QueryTemplate qt1 = application.QueryTemplates["qt1"];
52
Do not choose this option if: –You require shared access to the metadata –You require shared access to the streaming event data
53
StreamInsightHost.exe can be the server host
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.