1. Introduction to Azure Streaming Analytics
Warren Sifre
Introduction to Azure Streaming Analytics
April 4, 2017

2. Who am I? Warren Sifre Data Analytics Solution Architect Allegient
LinkedIn:

3. About me… In the IT Industry since 1998.
Developed system integration solutions against many different database platforms for various applications across many industries.
Passion in Solutions Architecture at both hardware and software levels.
Interests in SQL Server, MongoDB, Hadoop, Python/C#/PowerShell and Information Security (Hacking)
Indy BI PASS User Group Founder and Chapter Leader
PASS SQL Saturday / Indy PASS User Group presenter
MSCE: Data Platforms/Business Intelligence, Teradata 14 CTP and many more…

4. What is Streaming Analytics?
How it works?
Use Cases
Configuration and Dependencies
SAQL
Demo

5. Typical Environment

6. Challenges… Real-Time Analytics Environment Scalability
Many Steps in between the Source Data and the Visualization/Reporting Output.
Environment Scalability
Months/Years of planning is needed to plan out equipment procurement and scale out to meet increasing demand.
Resource Cost Management
Ideal configuration would require the purchasing of enough equipment to handle peak performance requirements. Although those peak requirements may only be for a few hours of any given day.
Disaster Recovery Strategy
Architecting and maintaining a DR strategy where performance, RTOs, and RPOs are met can be challenging and leave the organization with a lot of underutilized resources.

7. What is Streaming Analytics?
A way to evaluate data before it has reached its final repository destination.
Why?
Hours to weeks can be the time it takes for data to be transmitted, received, processed, aggregated, then visualized in the traditional Data Warehouse architecture. Business requirements have changed and the desire to glean insights from this data sooner is now becoming a requirement, not a nice to have.

8. How it can work… Streaming Analytics Job Event Hub Data Factory
Process and Deliver data to multiple end points
Transmit Data
Event Hub
Queues data for processing
Data Factory
Gather and process data for Predictive Analytics Process
Machine Learning
Predictive Analytics Processing
Power BI
Visualize real-time data stream
Azure SQL
Store data

9. Reimagined Environment

10. Use Cases Transportation Energy Manufacturing Medical Device
Reduce the need to pull vehicles from service for routine inspections by using sensors to determine when actual anomalies are occurring.
Energy
Monitor equipment from central locations such as Wind Turbines and Power Generators, thus reducing time spent on manual/physical inspection or replacement of parts just because of time instead of actual degraded performance.
Manufacturing
Monitor equipment and plant conditions for optimal performance.
Medical Device
Through remote monitoring expensive replacement parts can be ordered closer to the end-of- life of an equipment than by a schedule. This can reduce the cost of having an overstock of parts on-hand.

11. Configuration Options…
Add Input(s)
Data Stream
Event Hub
Blob Storage
IoT Hub
Reference Data
Add Query
Streaming Analytic Query Language (SAQL) - Similar to T-SQL
Add Output(s)
SQL Database
Blob Storage
Event Hub
Power BI
Table Storage
Service Bus Queue
Service Bus Topic
DocumentDB
Data Lake Store
Settings
Scale – How much processing power desired for SA Job?
Exception Handling– What is the definition of Late Data? What to do with late or out of order data?
Alerts – When do you want to receive a notification?
Functions – AzureML Integration

12. SAQL - Elements DML SELECT FROM WHERE GROUP BY HAVING
CASE WHEN THEN ELSE
INNER JOIN
LEFT OUTER JOIN
UNION
CROSS APPLY
OUTER APPLY
CAST
INTO
ORDER BY ASC, DSC
String Functions
Len
ConCat
CharIndex
Substring
PatIndex
Date and Time Functions
DateName
DatePart
Day
Month
Year
DateTimeFromParts
DateDiff
DateAdd
Aggregate Functions
Sum
Count
Avg
Min
Max
StDev
StDevP
Var
VarP
Windowing Extensions
TumblingWindow
HoppingWindow
SlidingWindow
Scaling Extensions
With
Partition By
Over
Temporal Functions
Lag
IsFirst
CollectTop

13. SAQL - Elements DML SELECT FROM WHERE GROUP BY HAVING
CASE WHEN THEN ELSE
INNER JOIN
LEFT OUTER JOIN
UNION
CROSS APPLY
OUTER APPLY
CAST
INTO
ORDER BY ASC, DSC
String Functions
Len
ConCat
CharIndex
Substring
PatIndex
Date and Time Functions
DateName
DatePart
Day
Month
Year
DateTimeFromParts
DateDiff
DateAdd
Aggregate Functions
Sum
Count
Avg
Min
Max
StDev
StDevP
Var
VarP
Windowing Extensions
TumblingWindow
HoppingWindow
SlidingWindow
Scaling Extensions
With
Partition By
Over
Temporal Functions
Lag
IsFirst
CollectTop

14. Tumbling Window
Fixed window of time with no overlap

15. Hopping Window
Fixed window of time with a fix time of overlap

16. Sliding Window
A Fixed window time, but a window is defined as the moment an event enters or exits an existing window.

17. Sliding Window
A Fixed window time, but a window is defined as the moment an event enters or exits an existing window.

18. Sliding Window
A Fixed window time, but a window is defined as the moment an event enters or exits an existing window.

19. Sliding Window
A Fixed window time, but a window is defined as the moment an event enters or exits an existing window.

20. Sliding Window
A Fixed window time, but a window is defined as the moment an event enters or exits an existing window.

21. Sliding Window
A Fixed window time, but a window is defined as the moment an event enters or exits an existing window.

22. Sliding Window
A Fixed window time, but a window is defined as the moment an event enters or exits an existing window.

23. Demonstration

24. Helpful Links More Information on Streaming Analytics SAQL
SAQL Query Patterns
Azure Portal Link
Azure Portal

25. Warren Sifre
LinkedIn: