11/30/2017 2:10 PM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN.

How to build IoT analytics in the real world
11/30/2017 2:10 PM B8055 How to build IoT analytics in the real world Santosh Balasubramanian, GPM Stream Analytics and Time Series Insights Shweta Gupta, Senior SDE TED © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Customer Scenarios Microsoft Build 2017 11/30/2017 2:10 PM
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Fighting Asthma Aerocrine makes medical devices that measure fractional exhaled nitric oxide (FeNO), a biomarker for airway inflammation. These instruments are used by thousands of physicians and nurses in hospitals and asthma clinics around the globe to identify asthma and monitor patients’ progress in controlling the disease. By using Azure Event Hubs as well as Azure Stream Analytics, Aerocrine can capture and manage real-time data about FeNO devices for the first time.

Smarter Buildings By collecting and processing real-time performance information from the equipment that regulates air quality and temperature in the facilities it services, Honeywell can monitor conditions, preempt maintenance, and save its global customers energy, time, and money.

Microsoft Build 2017 Transform your data into intelligent actions
11/30/2017 2:10 PM Asset Health Transform your data into intelligent actions © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Challenges Utilities Face
Microsoft Build 2017 11/30/2017 2:10 PM Challenges Utilities Face Utilities need to: Provide high reliability for consumers. Minimize risk to key assets. Minimize the cost of maintaining assets while optimizing reliability. One specific Challenge: Unexpected Substation Transformer failures can have a tremendous impact to reliability and cost. Risk Avoid  Manage Cost Minimize Optimize Performance Exceed Meet or beat © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Microsoft Build 2017 11/30/2017 2:10 PM The Solution Results A Neural Network Model was developed to predict failure state of a Transformer. The predictions provided warnings and critical alerts 8+ hours in advance of the failure, sufficient time to mitigate failure. The number of false alarms was minimal, no false critical alerts were generated. There does appear to be sufficient information in partial discharges to predict some failures. The Microsoft Azure platform simplified solution development. © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Scenario for this session
Microsoft Build 2017 11/30/2017 2:10 PM Scenario for this session Connect and Ingest Sensor data Real time monitoring – dashboarding and alerting Edge Analytics Time Series Insights* Predictive Analytics @AzureStreaming © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Azure Stream Analytics
Tech Ready 15 11/30/2017 Demo Architecture PowerBI IoT Hub Azure Stream Analytics Real-time dashboard Bluetooth Sensor Tag Presentation & Action Stream Analytics Event Queuing & Stream Ingestion Event Generation © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Connect and Ingest Sensor Data
Microsoft Build 2017 11/30/2017 2:10 PM Connect and Ingest Sensor Data @AzureStreaming © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Ideal world Process data Ingest data at regulator intervals
Microsoft Build 2017 11/30/2017 2:10 PM Ideal world Process data Ingest data at regulator intervals © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Real world Thousands of such devices that are generating data…
Microsoft Build 2017 11/30/2017 2:10 PM Real world Ingest data at regulator intervals Thousands of such devices that are generating data… © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Real world Thousands of different devices that are generating data
Microsoft Build 2017 11/30/2017 2:10 PM Real world Ingest data at regulator intervals Thousands of different devices that are generating data © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Microsoft Build 2017 11/30/2017 2:10 PM Real world Thousands of different devices that are generating data in different formats voltage Temperature, humidity Drag & Squeeze pressure Speed gauge Ingest data at regulator intervals © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Processing before ingestion
Microsoft Build 2017 11/30/2017 2:10 PM Processing before ingestion Connectivity with low powered/legacy devices Conversions (hex to int, voltage to liters) Compression (gzip) Collation from multiple low powered sensors Take real time actions © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Azure IoT Edge Microsoft Build 2017 11/30/2017 2:10 PM

Custom Bluetooth module
Microsoft Build 2017 11/30/2017 2:10 PM IoT Edge Modules IoT Hub client (built-in) Parser Module broker Custom Bluetooth module (custom) Stream Analytics © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Module Structure module.exports = { broker:null, configuration:null, …
11/30/2017 2:10 PM Module Structure module.exports = { broker:null, configuration:null, … create:function(broker,configuration){}, receive:function(message){}, start: function(){ broker.publish(new Uint8Array())}, destroy:function(){} } © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Gateway.json modules": [ { "name": “module1", "loader": {
11/30/2017 2:10 PM Gateway.json modules": [ { "name": “module1", "loader": { "name": "node", "entrypoint": { "main.path": “module1.js" } }, "args": {} }…] "links": [ "source": “module1", "sink": “module2" }, { "source": “module2", "sink": “module3" } © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Azure Iot-Edge Available on: https://github.com/azure/iot-edge
Microsoft Build 2017 11/30/2017 2:10 PM Azure Iot-Edge Available on: Language bindings Dotnet Dotnetcore Java Node.js © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Code Walkthrough Microsoft Build 2017 11/30/2017 2:10 PM

Real Time Analytics Nrtpmteam@Microsoft.com @AzureStreaming
Microsoft Build 2017 11/30/2017 2:10 PM Real Time Analytics @AzureStreaming © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Scenarios for Real Time Analytics
Microsoft Build 2017 11/30/2017 2:10 PM Scenarios for Real Time Analytics Automation Actions by Human Actors “See and seize” insights Live visualization Alerts and alarms Dynamic aggregation Machine to Machine Interactions Data movement with enrichment Kick-off workflows for automation Dashboarding Enriched Data Movement © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11/30/2017 2:10 PM PPT REM 3 Azure Stream Analytics Azure Stream Analytics Programmer Productivity Ease of getting started Fully managed Lowest TCO Mission critical reliability SQL like query language Source/sink integrations No cluster provisioning Pay as you go Enterprise grade SLA © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Powerful Analytics with Simple SQL Constructs
Data Manipulation SELECT FROM WHERE HAVING GROUP BY CASE WHEN THEN ELSE INNER/LEFT OUTER JOIN UNION CROSS/OUTER APPLY CAST INTO ORDER BY ASC, DSC Date and Time Functions DateName DatePart Day, Month, Year DateDiff DateTimeFromParts DateAdd String Functions Len Concat CharIndex Substring Lower, Upper PatIndex Temporal Functions Lag IsFirst Last CollectTop Mathematical Functions ABS CEILING EXP FLOOR POWER SIGN SQUARE SQRT Aggregate Functions SUM COUNT AVG MIN MAX STDEV STDEVP VAR VARP TopOne Windowing Extensions TumblingWindow HoppingWindow SlidingWindow Geospatial Functions (preview) CreatePoint CreatePolygon CreateLineString ST_DISTANCE ST_WITHIN ST_OVERLAPS ST_INTERSECTS Scaling Extensions WITH PARTITION BY OVER Link to Advanced Query Patterns

Real Time Analytics Tech Ready 15 11/30/2017 Presentation & Action
Storage & Batch Analysis Stream Analytics Event Queuing & Stream Ingestion Event production IoT Hubs Applications Archiving for long term storage/ batch analytics Real-time dashboard Automation to kick-off workflows Machine Learning Reference Data Event Hubs Blobs Devices & Gateways Data Lake, Document DB, SQL DB/DW, … PowerBI Service Bus Topics/Queues -> Worker Role … © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Demo Real Time Monitoring
Microsoft Build 2017 11/30/2017 2:10 PM Demo Real Time Monitoring @AzureStreaming © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Tech Ready 15 11/30/2017 Demo Architecture PowerBI IoT Hub Azure Stream Analytics Real-time dashboard Bluetooth Sensor Tag Presentation & Action Stream Analytics Event Queuing & Stream Ingestion Event Generation © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Additionally… Join multiple streams Reference data
Microsoft Build 2017 11/30/2017 2:10 PM Additionally… Join multiple streams Reference data Custom code with JavaScript Call out to Azure Machine Learning for real time scoring Built in troubleshooting and debugging capabilities Visual Studio plugin Scale out architecture through partitioning © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Temporal Semantics Application or ingest timestamp Windowing functions
Policies for event ordering Policies to manage latencies between ingress sources Manage streams with multiple timelines Join multiple streams on temporal windows Join streaming data with data at rest

Geospatial Functions CreatePoint CreatePolygon CreateLineString
ST_DISTANCE ST_WITHIN ST_OVERLAPS ST_INTERSECTS

Real-time Geospatial Analytics Scenarios
Phone Tracking Across Cell Sites Connected Car - Remote Management & Diagnostics Asset Tracking Fleet Management Facilities Management Personnel Tracking & Crowd Control Ride Sharing Geofencing Racecar Telemetry Connected Manufacturing Announcement and many more…

Microsoft Build 2017 11/30/2017 2:10 PM Geospatial Examples Generate an event when gas is less than 50 km from the car SELECT Cars.Location, Station.Location FROM Cars c JOIN Station s ON ST_DISTANCE(c.Location, s.Location) < 50 * 1000 Generate an event when fuel level is lower than 50%, a gas station is in promotion and course of car is pointing to gas station SELECT Cars.gas, Cars.Location, Cars.Course, Station.Location, Station.Promotion FROM Cars c JOIN Station s ON Cars.gas < 0.5 AND Station.Promotion AND ST_OVERLAPS(c.Location, c.course) Generate an event when a store is within a possible flooding zone SELECT Store.Polygon, Flooding.Polygon FROM Cars c JOIN Flooding f ON ST_OVERLAPS(s.Polygon, f.Polygon) Generate an event when a storm is heading my way SELECT Cars.Location, Storm.Course JOIN Storm s ON ST_OVERLAPS(c.Location, s.Course) Combination of clustering and heat maps. Clusters are represented using color coded geometric shapes that fit together evenly © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Edge Analytics with ASA
Microsoft Build 2017 11/30/2017 2:10 PM Edge Analytics with ASA @AzureStreaming © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Enables Industrial IoT Scenarios
Microsoft Build 2017 11/30/2017 2:10 PM Enables Industrial IoT Scenarios Low-latency command and control Systems such as manufacturing production lines or remote mining equipment need to analyze and act in real-time to the streams of incoming data. Sensor fusion on the edge Integrate together sensor from different systems Compliance Enables filtering or aggregation to remove PII data before sending it to the cloud Support of scenarios with intermittent connectivity Need of Resiliency: systems need to operate despite any interruption in the connectivity to the cloud. Local data reduction and transformation Goal is to transform raw input from sensors to meaningful information Enables scenarios with large volume of data Request access to the preview here © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

TechReady 23 HPe scenario 11/30/2017 2:10 PM On Premise Azure IoT Edge 2x Max-Fan can fans 3 speed switch 1 fan is off balance OPC UA or Modbus ingestion module Stream analytics on the Edge IOT Hub Module OPC UA Internet Ethernet NI cDAQ-9191 chassis NI-9234 module 4x Vibration sensors (X&Y axis) HPE ProLiant m510 Compute Node HPe is working on creating a differentiated “edge-as-a-service” offering for select vertical industry use cases. © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Demo ASA on Edge through Cloud Management
Microsoft Build 2017 11/30/2017 2:10 PM Demo ASA on Edge through Cloud Management © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Tech Ready 15 11/30/2017 Demo Architecture PowerBI Edge Analytics IoT Hub Azure Stream Analytics Real-time dashboard Bluetooth Sensor Tag Presentation & Action Stream Analytics Event Queuing & Stream Ingestion Event Generation © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Custom Bluetooth module
Microsoft Build 2017 11/30/2017 2:10 PM IoT Edge Modules IoT Hub client (built-in) Parser Module broker Custom Bluetooth module (custom) Stream Analytics © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Why is it differentiated?
A unique CEP engine – originally developed by MSR and already used on the cloud version of Azure Stream Analytics: High performance engine, sensor fusion SQL-like language and User Defined functions Ease of use for developers Same language for both Cloud and Edge “jobs”, helping customer to build hybrid architecture Cloud management: develop, deploy and manage from the cloud Multi-platforms: works on Linux and Windows Multi-protocols: leverage Azure IoT Edge SDK modules for connectivity to industry standards MQTT, OPC-UA, Modbus, Bluetooth LE, etc. Request access to the preview here

Time Series Insights Nrtpmteam@Microsoft.com @AzureStreaming
Microsoft Build 2017 11/30/2017 2:10 PM Time Series Insights @AzureStreaming © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Azure Time Series Insights Core Scenarios
11/30/2017 Azure Time Series Insights Core Scenarios IoT time series data visualization and analytics IoT validation and monitoring Unified view of multiple devices, plants, and data. Root cause analysis and anomaly detection Build custom applications using REST APIs © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Demo Time Series Insights
Microsoft Build 2017 11/30/2017 2:10 PM Demo Time Series Insights © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Azure Time Series Insights
Tech Ready 15 11/30/2017 Demo Architecture Applications IoT Hubs Azure Time Series Insights IoT Devices & Gateways Event Ingestion Time Series Analytics + Presentation & Action Event Exhausts © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Device Monitoring “Azure Time Series Insights has standardized our method of accessing devices’ telemetry in real time without any development effort. Time to detect and diagnose a problem has dropped from days to minutes. With just a few clicks we can visualize the end-to-end device data flow, helping us identify and address customer and market needs,” said Scott Tillman, Software Engineer, ThyssenKrupp Elevator.

Azure Time Series Insights
11/30/2017 2:10 PM Azure Time Series Insights Azure Time Series Insights is a fully managed analytics, storage, and visualization service that makes it incredibly simple to explore and analyze billions of events simultaneously. Get near real-time insights in seconds Start in seconds, scale in minutes Create a global view of your IoT-scale data Leverage the power of Time Series Insights in your Apps and Solutions T6026 (Data) : Time Series Insights on your IoT PM in Tech Talk A) © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Energy Demand Forecasting
Microsoft Build 2017 11/30/2017 2:10 PM Energy Demand Forecasting @AzureStreaming © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Demo Energy Forecasting
Build 2015 11/30/2017 2:10 PM Demo Energy Forecasting Link to demo © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Machine Learning & Data Science Conference
11/30/2017 2:10 PM Architecture Azure WebJob Runs jobs to get data from public source IoT Hub Streaming Data Stream Analytics processes events as they arrive in the EventHub Get Data Data Stream Job Data for Real-time Processing Energy Consumption Data & Weather Data (Public Source) Real time data stats Hourly Prediction Updates Power BI Dashboard Real Time Batch Send to Azure SQL for batch predictions Azure SQL Contains Historical Energy Consumption & Weather Data Azure Data Factory Pipeline invokes AML Web Service AML Model Web Service BES endpoint External Data Azure Services © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Appendix Nrtpmteam@Microsoft.com @AzureStreaming Microsoft Build 2017

Microsoft Build 2017 11/30/2017 2:10 PM Azure Stream Analytics © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Real Time Analytics Tech Ready 15 11/30/2017 Presentation & Action
Storage & Batch Analysis Stream Analytics Event Queuing & Stream Ingestion Event production IoT Hubs Applications Archiving for long term storage/ batch analytics Real-time dashboard Automation to kick-off workflows Machine Learning Reference Data Event Hubs Blobs Devices & Gateways Data Lake, Document DB, SQL DB/DW, … PowerBI Service Bus Topics/Queues -> Worker Role … © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

What is ASA Edge A Complex Event Processing (CEP) engine available on edge devices (e.g. Field Gateways) Able to run near-real-time analytics on multiple streams of data Simple operations: aggregates, projections, filters Complex operations: temporal-based joins, windowed aggregates, temporal filters Announcement

Azure Stream Analytics Differentiators
1,915 lines of code with Apache Storm @ApplicationAnnotation(name="WordCountDemo") public class Application implements StreamingApplication { protected String fileName = "com/datatorrent/demos/wordcount/samplefile.txt"; private Locality locality = null; @Override public void populateDAG(DAG dag, Configuration conf) { locality = Locality.CONTAINER_LOCAL; WordCountInputOperator input = dag.addOperator("wordinput", new WordCountInputOperator()); input.setFileName(fileName); UniqueCounter<String> wordCount = dag.addOperator("count", new UniqueCounter<String>()); dag.addStream("wordinput-count", input.outputPort, wordCount.data).setLocality(locality); ConsoleOutputOperator consoleOperator = dag.addOperator("console", new ConsoleOutputOperator()); dag.addStream("count-console",wordCount.count, consoleOperator.input); } } Programmer Productivity Declarative SQL like language Built-in temporal semantics Ease of Getting Started Integrations with sources, sinks, & ML Build real-time dashboards in minutes Lowest Total Cost of Ownership(TCO) Fully managed service No cluster topology management required Seamless scalability Usage based pricing 3 lines of SQL in Azure Stream Analytics SELECT Avg(Purchase), ScoreTollId, Count(*) FROM GameDataStream GROUP BY TumblingWindows(5, Minute), Score

Powerful Analytics with Simple SQL Constructs
Data Manipulation SELECT FROM WHERE HAVING GROUP BY CASE WHEN THEN ELSE INNER/LEFT OUTER JOIN UNION CROSS/OUTER APPLY CAST INTO ORDER BY ASC, DSC Date and Time Functions DateName DatePart Day, Month, Year DateDiff DateTimeFromParts DateAdd String Functions Len Concat CharIndex Substring Lower, Upper PatIndex Temporal Functions Lag IsFirst Last CollectTop Mathematical Functions ABS CEILING EXP FLOOR POWER SIGN SQUARE SQRT Aggregate Functions SUM COUNT AVG MIN MAX STDEV STDEVP VAR VARP TopOne Windowing Extensions TumblingWindow HoppingWindow SlidingWindow Geospatial Functions (preview) CreatePoint CreatePolygon CreateLineString ST_DISTANCE ST_WITHIN ST_OVERLAPS ST_INTERSECTS Scaling Extensions WITH PARTITION BY OVER

Mission Critical Reliability
Microsoft Build 2017 11/30/2017 2:10 PM Mission Critical Reliability Enterprise Grade SLA At least three 9s of availability Business Continuity During Failures Automatic checkpoint-recovery Fast restarts Guaranteed Event Delivery At-least-once event delivery semantics No data loss © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Global Availability Footprint
Microsoft Build 2017 11/30/2017 2:10 PM Global Availability Footprint Currently available in 19 Azure regions including China and Germany Current list includes: Central US, East US, East US2, North Central US, South, Central US, West US, North Europe, West Europe, East Asia, Southeast Asia, Japan West, Japan East, Brazil South, Australia East, Central India. In China: Stream Analytics is made available through a unique partnership between Microsoft and 21Vianet. In Germany: Stream Analytics is available via a new data trustee model whereby customer data remains in Germany under control of T-Systems, a Deutsche Telekom company, acting as the German data trustee. © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Satisfies Major Global Compliance Requirements
Microsoft Build 2017 11/30/2017 2:10 PM Satisfies Major Global Compliance Requirements Current list includes: ISO 27001 ISO 27018 SOC 1 Type 2 SOC 2 Type 2 SOC 3 Type 2 HIIPAA/HITECH PCI DSS Level 1 European Union Model Clauses China GB 18030 © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Advanced Query Examples
Microsoft Build 2017 11/30/2017 2:10 PM Advanced Query Examples More at © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Determine if a Value Has Changed
Microsoft Build 2017 11/30/2017 2:10 PM Determine if a Value Has Changed SELECT Make, Time FROM Input TIMESTAMP BY Time WHERE LAG(Make, 1) OVER (LIMIT DURATION(minute, 1)) <> Make Use LAG to peek into the input stream one event back and get the Make value. Then compare it to the Make on the current event and output the event if they are different © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Find First Event in a Window
Microsoft Build 2017 11/30/2017 2:10 PM Find First Event in a Window Find first car in every 10 minute interval SELECT LicensePlate, Make, Time FROM Input TIMESTAMP BY Time WHERE IsFirst(minute, 10) = 1 © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Find Last Event in a Window
Microsoft Build 2017 11/30/2017 2:10 PM Find Last Event in a Window Find last car in every 10 minute interval WITH LastInWindow AS ( SELECT MAX(Time) AS LastEventTime FROM Input TIMESTAMP BY Time GROUP BY TumblingWindow(minute, 10) ) Input.LicensePlate, Input.Make, Input.Time INNER JOIN LastInWindow ON DATEDIFF(minute, Input, LastInWindow) BETWEEN 0 AND 10 AND Input.Time = LastInWindow.LastEventTime There are two steps in the query – the first one finds latest timestamp in 10 minute windows. The second step joins results of the first query with original stream to find events matching last timestamps in each window. © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Detect Duration of a Condition
Tech Ready 15 11/30/2017 Detect Duration of a Condition Find out how long a condition occurred for. For example, suppose that a bug that resulted in all cars having an incorrect weight (above 20,000 pounds) – we want to compute the duration of the bug. WITH SelectPreviousEvent AS ( SELECT *, LAG([time]) OVER (LIMIT DURATION(hour, 24)) as previousTime, LAG([weight]) OVER (LIMIT DURATION(hour, 24)) as previousWeight FROM input TIMESTAMP BY [time] ) LAG(time) OVER (LIMIT DURATION(hour, 24) WHEN previousWeight < ) [StartFault], previousTime [EndFault] FROM SelectPreviousEvent WHERE [weight] < 20000 AND previousWeight > 20000 © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Microsoft Build 2017 SELECT System.Timestamp AS windowEnd, TopOne() OVER (ORDER BY t DESC) AS lastEvent FROM input TIMESTAMP BY t GROUP BY HOPPINGWINDOW(second, 300, 5) 11/30/2017 2:10 PM Fill Missing Values For the stream of events that have missing values, produce a stream of events with regular intervals. For example, generate event every 5 seconds that will report the most recently seen data point. SELECT System.Timestamp AS windowEndSELECT System.Timestamp AS windowEnd, TopOne() OVER (ORDER BY t DESC) AS lastEvent FROM input TIMESTAMP BY t GROUP BY HOPPINGWINDOW(second, 300, 5), TopOne() OVER (ORDER BY t DESC) AS lastEvent FROM input TIMESTAMP BY t GROUP BY HOPPINGWINDOW(second, 300, 5) This query will generate events every 5 second and will output the last event that was received before. Hopping Window duration determines how far back the query will look to find the latest event (300 seconds in this example). © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Detect Duration Between Events
Microsoft Build 2017 11/30/2017 2:10 PM Detect Duration Between Events Find the duration of a given event. For example, given a web clickstream determine time spent on a feature. SELECT [user], feature, DATEDIFF(second, LAST(Time) OVER (PARTITION BY [user], feature LIMIT DURATION(hour, 1) WHEN Event = 'start'), Time) as duration FROM input TIMESTAMP BY Time WHERE Event = 'end' © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Call to action Let attendees know what do to following this session.
11/30/2017 2:10 PM Call to action Let attendees know what do to following this session. Tell Attendees the URL where they can find your Github Code Use this slide to list resources, white papers, videos and links. Re-visit Build session recordings on Channel 9. Continue your education at Microsoft Virtual Academy online. #MSBuild © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11/30/2017 2:10 PM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN.

Similar presentations

Presentation on theme: "11/30/2017 2:10 PM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

11/30/2017 2:10 PM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN.

Similar presentations

Presentation on theme: "11/30/2017 2:10 PM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN."— Presentation transcript:

Similar presentations

About project

Feedback