1. Shubha Vijayasarathy
Program Manager, Azure Event Hubs - Microsoft
Azure Event Hubs: the world’s most widely used telemetry service

2. A Big Data Scenario Websites Devices Map reduce system
Real time monitoring
Data warehouse
Devices
Pipe to other system

3. A Big Data Scenario Azure Event Hubs Websites Devices
Map reduce system
Websites
Real time monitoring
Data warehouse
Devices
Pipe to other system

4. Big Data Design Elements
Design considerations, “How do we…”
Azure Event Hubs
handle variety?
handle volume?
handle variety?
Heterogeneous inputs - devices, web pages, VMs, Docker containers, multi-language clients
handle volume?
Huge amount of data needs to be ingested
handle high velocity?
Data delivered at different speeds
scale out/in on demand?
Workloads increase during a busy week
Event Hubs Auto-Inflate
handle high velocity?
scale out/in on demand?
Auto-Inflate

5. Big Data Design Elements
Design considerations, “Can we…”
Azure Event Hubs
…regional disaster?
GeoDR
do near real-time stream and batch?
Capture
…regional disaster?
Event Hubs GeoDR
do near real-time stream and batch
Capture for batch
estimate costs accurately?
Simple billing model
estimate costs accurately?
Simple billing model
rapidly move from proto  prod?

6. Big Data Design Elements
Design considerations, “How do we…”
Azure Event Hubs
get support?
use the best solutions?
Event Hubs - world class team behind you!
get world class support
Fully managed service with SLAs. Huge community support too.
use best solutions
You don’t have to be big data experts
Event Hubs - world class team behind you who are working on bleeding edge of big-data + streaming technology

7. Big Data Design Elements
Design considerations, “Integrate story…”
Azure Event Hubs
Integrate?
Simplified integration with
Azure Event Grid
Azure Stream Analytics
Logic Apps Event Hubs connector
Azure Functions
Apache Spark
Apache Storm
Apache Nifi …
easily integrate with other systems
Simplified integration with
Azure Event Grid
Azure Stream Analytics
Logic Apps Event Hubs connector
Azure Functions
Apache Spark
Apache Storm
Apache Nifi …
Many more…

8. Big Data Design Elements
Design considerations, “Do we…”
Azure Event Hubs
manage the service?
diagnose and monitor?
manage the service?
Complete PaaS solution, we manage the OS upgrades, patching, you just use the service
Integrated with Azure Monitor for alerting and monitoring
Azure Monitor for Event Hubs metrics
Data governance
Compliant services -
follow data governance practices?

9. Azure Event Hubs for Big Data
Event producers
Collection
Ingress
Stream Processing
Long-term storage
Presentation / action
Applications
Fast Data
Service bus
Azure DBs
HDInsight
Azure Storage
Search and query
Cortana Analytics
PowerBI Dashboards
Cloud gateways (web APIs)
Event hubs
Legacy IOT (custom protocols)
Stream processing
IP-capable systems (containers/VMs/servers)
Slow Data
Devices
Azure Data Lake
Field gateways
Low-power devices (RTOS)
Devices to take action

10. Event Hubs  Data Warehouse - using Event Grid (Demo)
Stream big data into a data warehouse
GitHub sample
Deploy the infrastructure with this sample template.

11. Coming soon! Azure Geo DR General Availability IP filtering
VNet integration
Dedicated clusters